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UNIFORM BOUNDS FOR NORMS OF SUMS OF INDEPENDENT RANDOM 

FUNCTIONS 

By Alexander Goldenshluger * and Gleg Lepski 

University of Haifa and Universite Aix-Marseille I 

In this paper we develop a general machinery for finding explicit 
uniform probability and moment bounds on sub-additive positive 
functionals of random processes. Using the developed general tech- 
nique we derive uniform bounds on the Ls-norms of empirical and 
regression-type processes. Usefulness of the obtained results is illus- 
trated by application to the processes appearing in kernel density 
estimation and in nonparametric estimation of regression functions. 



1. Introduction. 

1.1. General setting. Let & and be linear topological spaces, (17,21,?) be a complete proba- 
bility space, and let ^ : x — )• © be a linear and continuous on S) mapping. Let \I' : © — )■ M+ be 
a given suh-additive functional. Suppose that there exist functions A : Sj ^ M_|_, i? : f) — )• ]R_|_ and 
U ■.S)^R+ such that 

(1.1) p{«i.fe)-cfw>4<i,( ^,^,);'^(,)J . w.ft 

where g : M+ — >■ M+ is a monotone decreasing to zero function. 

Let G be a fixed subset of S). In this paper, under rather general assumptions on U, A, B and Q, 
we establish uniform probability and moment bounds of the following type: for any e € (0, 1) and 
y >0 

(1.2) P| sup \^{ie) - (1 + + yXs) U{d)\ > ol < P,,g{y), 

(1.3) ^snv\^{ie)-u,[l + ^\A + y\B)U{e)X' < E,^g{y), q>l. 

Here Xa and are the quantities completely determined by C/, A, Q and U,B,@ respectively, and 
the inequalities (1.2) and (1.3) hold if these quantities are finite; Pe,g{ ) and Ee^g{-) are continuous 
decreasing to zero functions completely determined by e and g; and the factor is such that 

— )■ 1, e — )• 0. We present explicit expressions for all quantities appearing in (1.2) and (1.3). 

In order to derive (1.2) and (1.3) from (1.1) we assume that the set G is the image of a totally 
bounded set in some metric space under a continuous mapping. Namely, if (3,d) is a metric space, 
and Z is a totally bounded subset of (3, d) then we assume that there exists a continuous mapping 
(j) from 3 to such that 

(1.4) G = {eGiO :^ = 0[C], CeZ}. 
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Let Ni,d{S), S > he the minimal number of the balls in the metric d needed to cover Z. The 
inequalities (1.2) and (1.3) are proved under some condition that relates N^^di') and g{-). It is 
worth mentioning that in particular examples the parametrization Q = (^[Z] is often natural, while 
the metric d may have a rather unusual form. 

The inequalities (1.2) and (1.3) can be considered as a refinement of usual bounds on the tail 
distribution of suprema of random functions. In particular, probability and moment bounds for 
supggQ ^(Ce) can be easily derived from (1.2) and (1.3). The well-known concentration results deal 
with deviation of the supremum of a random process from the expectation of this supremum, and 
estimation of the expectation is a separate rather difficult problem. In contrast, in this paper we 
develop explicit uniform bounds on the whole trajectory {^{^0),O G Q}. The inequality in (1.1) 
provides the basic step in the development of such uniform probability bounds. The usual technique 
is based on the chaining argument that repeatedly applies the inequality in (1.1) to increments of the 
considered random process [see, e.g., Ledoux and Talagrand (1991) and (van der Vaart and Wellner 
1996, Section 2.2)]. 

The most interesting phenomena can be observed when a sequence of random mappings | ^^"^ , 9 G 
.^}, n G N* is considered. There exists a class of problems where the quantities A^^ and depend 
on n, and A^ — )• 0, A^ — )■ as n — )• oo. Under these circumstances, one can choose y = y„ — )• oo 
and e = e„ — )■ such that 

(1.5) Pe,g{yn) ^ 0, E^,giyn) ^0, ?W OO. 

and, at the same time, 

(1.6) Ue„{l + ^/lMXA + yn>^B)U{■) ^ U{-), n OO. 

The relation in (1.5) means that iie„(l + ^/y^)^A + yn^B)U{-) is indeed a uniform upper bound 

for ^'(,^g"^) on 0, while (1.6) indicates that for large n this uniform bound is nearly as good as 
a non-uniform bound [/(•) given in (1.1). Typically for a fixed y > we have Pe^g{y) — )• oo and 
Ee,g{y) — )• oo as e — )■ 0; therefore, in order to get (1.5) and (1.6), e„ — and 2/„ — oo should be 
calibrated in an appropriate way. 

The general setting outlined above includes important specific problems that are in the focus 
of the present paper. We consider sequences of random mappings that are sums of real-valued 
random functions defined on some measurable space (here the parameter n G N* is the number of 
summands). We are interested in uniform bounds on the norms of such random functions; thus the 
sub-additive functional of interest \E' is the L^-norm, s > 1. First the non-uniform bound (1.1) is 
established, and then the inequalities of the type (1.2) and (1.3) are derived. It is shown that (1.5) 
and (1.6) hold under mild assumptions on the parametric set B. We also discuss sharpness of the 
non-uniform inequality in (1.1). 

1.2. Norms of sums of independent random functions. Let {T,^,t) and (Af,X, z^) bee-finite 
spaces, and let X he a. separable Banach space. Consider an Af-valued random element X defined 
on the complete probability space (ri,2l, P) and having the density / with respect to the measure 
v. Let e be a real random variable defined on the same probability space, independent of X and 
having a symmetric distribution. 

For any (T x X) -measurable function w onT ^ X and for any t G T, n G N* , define the random 
functions 

n n 

(1.7) i^{t) [u;(t,X,) -Eu;(t,X)] , ?7^(t) ■.= Y,w{t,Xi)ei, 

1=1 i=l 

2 



where (Xj, ej), i = 1, . . . ,n, are independent copies of (X, e). Put for 1 < s < oo 



\Ut)\'r{dt) 



l/s 



\vUtrr{dt) 



l/s 



We are interested in uniform bounds of the type (1.2)-(1.3) for ||^i„||s,t and ||?/u,||s,r when w e W, 
where W is a given set of (T x X)-measurable functions. This setup is a specific case of the general 
framework with ^{■) = \\ ■ \\s,ti ^ = w and = W. More precisely, if ip^ denotes either or rj^, 
and if P is the probability law of Xi, . . . , Xn (when is studied) or of {Xi,ei), . . ., {Xn,£n) (when 
r]^ is studied) then we want to find a functional U^ip^j) = U^{w,f) such that (1.1) holds and 



(1.8) 
(1.9) 



sup 



E sup 



-'W \\S,T 



u,{l + ^XA + yXB)U^{wJ)\ >o| < P,,g(y), 
'X,{l + ,/^\A + yXB)U^{wJ)y < E,^g{y), q>l 



Note that {^w,u} G W} is the empirical process. In the sequel we refer to {'i]w,w G W} as the 
regression-type process as it naturally appears in nonparametric estimation of regression functions. 
In the regression context Xi are the design variables while £i are the random noise variables. 

Uniform probability and moment bounds for empirical processes are a subject of vast literature, 
see, e.g., Alexander (1984), Talagrand (1994), van der Vaart and Wellner (1996), Massart (2000), 
Bousquet (2002), Gine and Koltchinskii (2006) among many others. Such bounds play an important 
role in establishing the laws of iterative logarithm and central limit theorems [see, e.g., Alexander 
(1984) and Gine and Zinn (1984)]. However we are not aware of works studying uniform bounds of 
the type (1.8)-(1.9) satisfying (1.5) and (1.6) for the L^-norms of such processes. 

Apart from the pure probabilistic interest, development of uniform bounds on the L^-norms 
of processes {(,^^,1^ € W} and {rjyj,w G W} is motivated by problems of adaptive estimation 
arising in nonparametric statistics. In particular, the processes € W} and {rj^jW G W} 

represent stochastic errors of linear estimators with the weight w in the density estimation and 
nonparametric regression models respectively. Uniform bounds on the error process are key technical 
tools in development of virtually all adaptive estimation procedures [see, e.g., Barron et al. (1999), 
Devroye and Lugosi (2001) Cavaher and Golubev (2006), Goldenshluger and Lepski (2008, 2009), 
and Golubev and Spokoiny (2009)]. 

The kernel density estimator process is a particular case of the empirical process {S.wi'w € W} 
that was frequently studied in the probabilistic literature. It is associated with the weight function 
w given by 



(1.10) 



w{t, x) 



1 



n Yli=i hi 



-K 



X 



h 



xeX 



t G r 



where : R — )• M is a kernel, h = {hi, . . . ,hd) is the bandwidth vector, and u/v denotes the 
coordinate-wise division for u,v G W^. Limit laws for the L<j-norms of the kernel density es- 
timators were derived in Beirlant and Mason (1995); Diimbgen and Fatalov (2002) study exact 
asymptotics for the large/moderate deviation probabilities. Gine, Mason and Zaitsev (2003) inves- 
tigate weak convergence of the Li-norm kernel density estimator process indexed by a class of 
kernels under entropy conditions. For other closely related work see Einmahl and Mason (2000), 
Gine, Koltchinski and Zinn (2004), Gine and Nikl (2008) and references therein. We remark that 
the kernel density estimator process is naturally parametrized hy W = K. x Ti, where is a set 
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of band widths and /C is a family of kernels. However, the distance d induced on /C x 7^ is rather 
special, see Section 3.4. The convolution kernel density estimator process will be also studied in 
Section 3.4. 

The inequalities (1.8)-(1.9) are useful for constructing statistical procedures provided that the 
following requirements are met. 

(i) Explicit expression for U^{w,f). Typically the bound U^,{w,f) is directly involved in the 
construction of statistical procedures; thus it should be explicitly given. 

(ii) Minimal assumptions on W. This condition is dictated by a variety of problems where the 
inequalities (1.8)-(1.9) can be applied. In particular, the sets W may have a complicated 
structure [see, for instance, examples in Section 3.4]. 

(iii) Minimal assumptions on f. The probability measure P (and the expectation E) as well as 
the right hand sides of (1.8)-(1.9) are determined by the density /. Therefore we want to 
establish (1.8)-(1.9) under weak assumptions on /. In particular, we would like to emphasize 
that all our results are established for the set of all probability densities uniformly bounded 
by a given constant. No regularity conditions are supposed. 

(iv) Minimal assumptions on the distribution of e. If the process {r]yi],w £ W} is considered, then 
the probability measure P (and the expectation E) is also determined by the distribution of 
e. Therefore we would like to have (1.8)-(1.9) under mild assumptions on this distribution. 
We will see that the function g given in (1.1) depends on the distribution tail of e. 

Let us briefly discuss some consequences of the requirement (i) for the process {^wjW € W}. 
Using the Talagrand concentration inequality we prove that (1.1) holds with U^{w,f) = EII^^uH^^t-, 
on the space of functions S) = {lu : sup^^;^ \\'^{'^^)\\s,t < oo}. However this bound cannot be used 
in statistical problems at least for two reasons. 

Firstly, it is implicit, and a reasonably sharp explicit upper bound U^{w,f) on U^{w,f) should 
be used instead. Sometimes if the class W is not so complex (e.g., W = /C x T-L) one can find a 
constant c independent of w, f and n such that 

cUi:{w,f)<U^{wJ)<U^{wJ). 

In such cases U^{w,f) can be regarded as a sharp bound on U^{w,f). We note, however, that 
establishing the above inequalities requires additional assumptions on W and / and requires non- 
trivial technical work. It seems that for more complex classes W the problem of finding an "optimal" 
upper estimate for U^{w, f) cannot be solved in the framework of probability theory. Contrary to 
that, the theory of adaptive nonparametric estimation is equipped with the optimality criterion, and 
an upper bound U^{w, f) can be regarded as sharp if it leads to the optimal statistical procedure. 
Thus sharpness of U^{w, f) can be assessed through the accuracy analysis of the resulting statistical 
procedure. 

Secondly, U^{w, f) (and presumably its sharp upper bound U^{w, /)) depends on /. In the den- 
sity estimation context where the process {iw,w G W} appears, / is the parameter to be estimated. 
Therefore bounds depending on / cannot be used in construction of estimation procedures. A nat- 
ural idea is to replace U^{w, f) by its empirical counterpart U^{w) whose construction is based only 
on the observations Xi, . . . , Xn. We adopt this strategy and establish the corresponding inequality 

(1.11) E sup \\\U\s,r-v,{l + ^XA + y\B)M'^)y < Kgiv)^ 9>1, 

where E^^g{-) differs from E^^g{-) in (1.9) only by some absolute multiplicative factor, and, there- 
fore, satisfies (1.5) if (1.6) holds for U^{w,f). Here Ve is bounded by some absolute constant and 
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completely determined by e and W. We provide an explicit expression for v^- 

Thus the requirement (i) leads to a new type of uniform bounds that are random. A natural 
question about sharpness of these bounds arises. In order to give an answer to this question, we 
prove that under mild assumptions on the class of weights W one can choose e = — )• and 
y„ — 7- oo as n — )■ oo so that 



Ver^ (l + Vyn^A + ynAfi) = ! + Hm £„ = 0, lim E^^g{yr, 

and for any subset Wq ^ W and any q > 1 



0, 



E 



sup U^{w) 



< 



(l + in) sup U^iwJ) +Rn{m), 



where the remainder term i2,„(Wo) is asymptotically negligible in the sense that for any £ > 
one has limsup^^o^ supjgj- supyy^jcw [^^-^n(Wo)] = 0. Here T denotes the set of all probability 
densities uniformly bounded by a given constant [see (3.8)]. These results show that in asymptotic 
terms the random uniform bound is almost as good as the non-random one, and there is no loss of 
sharpness due to the use of the random uniform bound. 



1.3. Summary of results and organization of the paper. In this paper we develop a general 
machinery for finding uniform upper bounds on sub-additive positive functionals of sums of inde- 
pendent random functions. We start with the general setting as outlined in Section 1.1 above, and 
establish inequalities of the type (1.2) and (1.3) [see Proposition 2]. Proofs of these results are based 
on the chaining and slicing/peeling techniques. The distinctive feature of our approach is that Q 
is assumed to be an image of a subset Z, of a metric space under some continuous mapping (j), i.e. 
Q = (j){'L) as in (1.4). Then chaining on is performed according to the distance induced on by 
the mapping (j). 

Section 3 is devoted to a systematical study of the L^-norm of the empirical process {iw^ w G W}. 
First we derive an inequality on the tail probability of ||.^«;||s,t for an individual function G W [see 
Theorem 1 in Section 3.1]. Here we use the Bernstein inequality for empirical processes proved by 
Bousquet (2002) and inequalities for norms of integral operators. Then in Section 3.2 we proceed 
with establishing uniform bounds. In Theorem 2 of Section 3.2.1 we derive uniform non-random 
bounds for ||^ui|U,ti w G W that hold for all s > 1. In the case s > 2 the non-random bound 
depends on the density /; therefore for s > 2 we construct a random bound and present the 
corresponding result in Theorem 3. Theorems 2 and 3 hold for classes of weights W satisfying 
rather general conditions. In Section 3.3 we specialize results of Theorems 2 and 3 to the classes 
W of weights depending on the difference of their arguments. This allows to derive explicit both 
non-random and random uniform bounds on ||^«,||s,t under conditions on the weights which can be 
easily interpreted. The corresponding results are given in Theorems 4 and 5. We also present some 
asymptotic corollaries which demonstrate sharpness of the derived uniform bounds. Section 3.4 
applies the results of Theorems 4 and 5 to special examples of the set W. In particular, we consider 
the kernel density estimator process given by (1.10), and the convolution kernel density estimator 
processes. It turns out that corresponding results can be formulated in a unified way, and they are 
presented in Theorem 7. 

In Section 4 we study L^-norm of the regression-type processes {?/«,, w G W} given in (1.7). 
First we derive an inequality on the tail probability of ||??iu||s.r for an individual function it; G W 
[Theorem 8 in Section 4.1]. This theorem is proved under two different types of conditions on the 
tail probability of the random variable e. In Section 4.2 we present a non-random uniform bound for 
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||??«)||^ ^ for all s > 1 over the class of weights depending on the difference of their arguments. The 
corresponding result is given in Theorem 9, and some asymptotic results that follow from Theorem 9 
are formulated in Corollary 7. Sections 5-10 contain proofs of main results of this paper. Proofs of 
auxiliary lemmas are given in Appendix. 

2. Uniform bounds in general setting. In this section we establish uniform probability 
bounds for the supremum of a general sub-additive functional of a random process from the prob- 
ability inequality for the individual process. 

Let & and Sj be linear topological spaces, and (Q,2t, P) be a complete probability space. Let 
\I' : 6 — >• be a given sub-additive functional, and B be a given subset of S). We consider the 
mapping ,^ : x O — )• (3 such that 

• ^, is linear and continuous on S) P-almost surely; 

• is 2t- measurable for any 6 £ Sj. 

Assumption 1. There exist functions A : ^ B : fj ^ U : Sj ^ and : M+ M+ 
such that 

(i) for any z > 

(ii) the function g is monotonically decreasing to 0; 

(iii) < r := inf U{e) < supU{9) =: R < oo. 

6»G0 6ig0 

Condition (i) is a Bernstein-type probability inequality on ^{S,g) for a fixed 9 £ Sj. Based on 
Assumption 1, our goal is derive uniform probability and moment bounds of the type (1.2) and 
(1.3). For this purpose we suppose that the set © is parametrized in a special way; this assumption 
facilitates the use of the standard chaining technique and leads to quite natural conditions on the 
functions U, A and B. 

Assumption 2. Let (3,d) be a metric space, and let Z be a totally bounded subset of (3jd). 
There exists a continuous mapping (j) from to such that 

e = {6 € : e = (bid Cez}- 

Remark 1. In statistical applications the set is parametrized in a natural way. For instance, 
if, as in the introduction section, ^(•) = || • \\s^r CLnd = S^yj with w given by (1.10), then B is 
parametrized by the kernel and bandwidth (K, h) £ IC x H. The distance d on IC x H may have a 
rather special form. 

Let Z he a subset of Z. Define the following quantities: 

(2.1) xu[Z) := sup V sup [/((/>[(]), 

(2.2) Aa{Z) := sup V supA(0[C]), 

^OQ\ A ^'7\ B{(f>[(l]- (f>[C2]) R^-FAIN 

(2.3) Ab{Z) := sup V sup B{(j)[Q). 



Let Nz^di^) denote the minimal number of balls of radius 6 in the metric d needed to cover the set 
Z, and let £z,d{S) = In [-^z,d(5)] be the 5-entropy of Z. For any y > and e > put 



44(y) = 5(2/) + EKd(62-^') 



fc-3, -2 



9 Qy^'-'k 



k=l 

Key propositions. The next two statements are the main results of this section. Define 

(2.4) C*{y,Z):=^AA{Z)+yAB{Z), y > 0, 
where and are given in (2.2) and (2.3). 

Proposition 1. Suppose that Assumptions 1 and 2 hold, and let Z he a subset ofL such that 
sup^^^/g2 d(C, CO ^ and >cij{Z) < oo. Then for all y > and e > one has 

p| sup *(^^[^]) > (1 + 6) [MZ) + C*{y, Z)] I < 4';^(y). 

Remark 2. Inspection of the proof of Proposition 1 shows that continuity of ^, on f) can be 
replaced by the assumption that is continuous P -almost surely on (j)[7j] in the distance d. The 

latter assumption is often easier to verify in specific problems. 

Define 

(2.5) *:(y,Z):=sup{*(e^[^])-nC*(y)t/(</.[C])}, y > 0, 

where Z C Z is a subset of Z, u > 1 is a constant, and C*(-) is the function defined below in 
(2.9). We derive bounds on the tail probability and q-th moment of the random variable \I'^(y,Z). 
Note that ^*(y, Z) is 2l-measurable for given y and u because the mapping ( i— )• is P-almost 
surely continuous, and Z is a totally bounded set. By the same reason the supremum taken over 
any subset of Z will be measurable as well. 

With r and R defined in Assumption l(iii), for any a G [r, R] consider the following subsets of Z: 

(2.6) Z, := {C G Z : a/2 < C/(,^[C]) < a} . 

In words, for given a G [r, R], Z^ is the slice of the parameter values C £ Z for which the function 
U{4i[(]) takes values between a/2 and a. 

In what follows the quantities X[/(Za), A^(Za), AB(Za) and L^g\^{y) will be considered as 
functions of a G [r, R\. That is why, with slight abuse of notation, we will write 

(2.7) xu{a) := M^a), L^\y.a) := ^^(l/). 
Put also 

(2.8) Ka:= sup a-UA(Za); Ab := sup o"Uij(Za), 

aG[r,i?] ae[r,i?] 

and let the function C*(-) in (2.5) be defined as 

(2.9) C*{y):=l + 2^kA + 2yKB. y > 0. 
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Proposition 2. Suppose that Assumptions 1 and 2 hold, and let >cu{'L) < oo. If 
(2.10) xc/(a) <a, Va G [r,R], 

and if Ue = 2*^(1 + e) then for any e € (0, 1], y > and any g > 1 one has 



(2.11) 



(2.12) 



£-ilog2 {R/r)-l 

P{KAy,^) > 0} < iVz,d(6/8) Yl 4'\y,r2<^+'^), 

j=0 



E 



€-llog2 (RA)-i 



where Jg^\z,a) := q {x — 1)'^ ^L)j''{zx,a)dx. 
Remark 3. 

1. Proposition 1 establishes an upper hound on the tail probability of the supremum of '^{^^[c^]) 
over an arbitrary subset of Z contained in a ball of radius e/8 in the metric d. The proof of 
Proposition 2 uses this bound for balls Z of the radius e/8 that form a covering ofTL. Each ball 
Z is divided on slices on which the value ofU{(p[C]) is roughly the same. Then the supremum 
over TL is bounded by the sum of suprema over the slices. This simple technique is often used 
in the literature on empirical processes where it is referred to as peeling or slicing [see, e.g., 
(van de Geer 2000, Section 5.3) and Gine and Koltchinskii (2006)]. 

2. Note that Proposition 2 holds for any distance d on 3- Therefore if xu{a) is proportional to 
a, the condition (2.10) can be enforced by rescaling the distance d. 

We now present a useful bound that can be easily derived from (2.11) and (2.12). Let 

oo 

:= j;[iVz,d(62-'=)]V5(9-2^-='^-')- 

k=l 



^* (y,Z) <iVz,d(e/8) [u,C*{y)f 



^ [,2<.+i)]VW(y,,2<.+i)), 

j=0 



(2.13) 



Remark that for all Z C Z and y > 1 



L%{y)<g{y) + Lf^). 



because inffc>i 2^{k) ^ = 8/9 and g is monotone decreasing. Therefore we arrive to the following 
corollary to Proposition 2. 

Corollary 1. // assumptions of Proposition 2 hold, and L^g^ < oo then for all y > 1 and 
e G (0, 1] 



P{KSy,^)>0} < N^a{^ / S) e-Hog^ {R/ r) g{y)+Lf J g{y 



E 



*:^(y,Z)J^ < iV2;,d(e/8) [22^i?(l + e)C*(y)]'^[2'^^-l]^V(^)(y), 
where Jg^\z) = q J^{x — 1)'^^^ g{zx) + L^g\ g(yZx) 



dx. 



3. Uniform bounds for norms of empirical processes. Based on the results obtained in 
Propositions 1 and 2, in this section we develop uniform bounds for the family "u^ G VV}, 

where S^^j is defined in (1.7). The first step here is to check Assumption 1. For this purpose we 
establish an exponential inequality for \\(,w\\ when the function w W is fixed. Next, using 
Corollary 1 we derive a non-random uniform bound and establish corresponding inequalities of the 
type (1.8)^(1.9) satisfying requirements (i)-(iv) of Introduction. We develop also a random uniform 
bound based on Xi, . . . , Xn and derive an inequality of the type (1.11). 

To proceed we need the following assumption. 

Assumption (Al). Let X he the countable dense subset of X . For any e > and any x £ X 
there exists x € X such that 

\\w{-,x) - w{-,x)\\s,T < £■ 

In the sequel we consider only the sets W of (T x X)-measurable functions satisfying Assump- 
tion (Al). Let 

v'{(ix) = /(x)z^(dx), 

and for any s G [l,oo] define 

1/. 



\w{tr)\\%,.r{dt) 



l/s 



s/2 

\w{t,x)\'f{x)u{dx)] r(dt) 



(3.1) Ms^^y{w) := sup\\w{-,x)\\s,r V sup ||t(;(t, •) Us,;.' , Ms{w) := Ms,r,u{w)- 

Let ci(s) := 15s/ Ins, s > 2, C2(s) be the constant appearing below in the inequality (6.2) of 
Lemma 3, and define 



(3.2) 



C3(s) := ci(s) V C2(s/(s - 1)), Vs > 2, c*(s) := < 



' 0, 1 < s < 2, 
1, s = 2, 

, C3(s), s > 2. 



It is worth mentioning that ci(s) is the best known constant in the Rosenthal inequality [see Johnson et al. 
(1985)], and in many particular examples C2(s) = 1 [see Lemma 3 below]. Although ci(s) is defined 
for s > 2 only, it will be convenient to set ci(s) = 1 if s G [1,2]. We use this convention in what 
follows without further mention. 

3.1. Probability bounds for fixed weight function. For any w £ W we define 

' [V^j:,iw, /)] A [An^/'Ms{w)] , s < 2, 

(3.3) PsiwJ) := I V^M2iw), s = 2, 

^ ci{s)[^^s{wJ) + 2n^/'Ms{w)], s>2. 



and if s > 2 then we set 
(3.4) u:l{wJ):=Qc^{s) 



M2(w)[l4n + 96ni/'^] s < 2, 

6 nMl ^ ^, {w) + 24 VnM| {w) s = 2, 



n 
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s+2 ' 



Theorem 1. Let s G [l,oo) be fixed, and suppose that Assumption (Al) holds. If Ms{w) < oo 
then for any z > 

(3.5) r{\\Us,r>Psiw,f) + z} < expj- ^ f I , 

I 5^^i(^«,/) + ic*(s)M^(u;)z J 

where c*(s) is g'wen m (3.2). 

Remark 4. Because c*(s) = /or s G [1,2), the distribution of the random variable ||^io||s.r 
has a sub-Gaussian tail. In this case similar bounds can be obtained from the inequalities given in 
(Pinelis 1990, Theorem 2.1), (Pinelis 1994, Theorems 3.3-3.5) and (Ledoux and Talagrand 1991, 
Section 6.3). In particular. Theorem 1.2 of Pinelis (1990) gives the upper bound ex.p{— z'^ /2nMg (w)} 
which is better by a constant factor than our upper bound in (3.5) whenever s £ [1,2). However, if 
s > 2 then the cited results are not accurate enough in the sense that the corresponding bounds do 
not satisfy relations (1.5)-(1.6) of Introduction. It seems that only concentration principle leads to 
tight upper bounds; that is why we use this unified method in our derivation. 

It is obvious that the upper bound of Theorem 1 remains vaUd if we replace Ps{w, /), u}1{w, f) 
and Ms{w) by their upper bounds. The next result can be derived from Theorem 1 in the case 
[1,2). 

Corollary 2. Let s G [1,2) be fixed, and suppose that Assumption (Al) holds. If Ms{w) < oo 
then for every z > and for all n > 1 

HWU.r > 4n"-M,(») + 4 < exp { - ^^^} ■ 

The result of the corollary is valid without any conditions on the density /. Moreover, neither 
the bound for ||^u)||s,r, nor the inequality right hand side depend on /. It is important to realize 
that the probability inequality of Corollary 2 is sharp in some cases. In particular, it is not too 
difficult to construct a density / such that T,siw,f) = +oo for any function w satisfying rather 
general assumptions. In this case the established inequality seems to be sharp. On the other hand, 
for any density / satisfying a moment condition y/nTis{w, f) can be bounded from above, up to a 
numerical constant, by ^/nM2{w) which is typically much smaller than n^/'^Ms{w). 

Several useful bounds can be derived from Theorem 1. In particular, it is shown at the end of 
the proof of Theorem 1 that for all s > 2 and p > 1 

(3.6) ^s{w,f) < M2{w)\\y^\\s,u, Mp^ryiw) < [1 V ll/lloo]'/^ Mp{w). 
Using these inequalities we arrive to the following result. 

Corollary 3. Let s > 2 be fixed, and suppose that Assumption (Al) holds. If Ms{w) < oo 
then for every z > and for all n > 1 

( 1 



\ul{w,f)-r\c^{s)M,{w)z 



P{||^«,||s,r >Ps(w',/)+^} < expj 

where Psiw.f) := ciis)[^M2{w)\\^/f\\s,u + 2n'^/'Ms{w)] and 

Co^,{w, f) := 6c3(s){n[l V ||/||oo] M%^{w) + AV^M2{w)M,{w)\\y^\U,, + Sn^^'M^iw)}. 
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3.2. Uniform bounds. Theorem 1 together with Corollaries 2 and 3 ensures that Assumption 1 
is fulfilled for ||^«,||s,r- Ii^ this section we use Proposition 2 together with Theorem 1 in order to 
derive a uniform over W bounds on ||^«,||s.r- 

Following the general setting of Section 2 we assume that W is a parametrized set of weights, 
i.e., 

(3.7) W = {w:w = ct>[(], C€Z}, 

where Z is a totally bounded subset of some metric space (3, d). Thus any w £ W can be represented 
as w = (plC] for some ^ € Z. Recall that Ni^di^)^ S > stands for the minimal number of balls of 
radius 5 in the metric d needed to cover the set Z, and £z,d{S) = ^^[Nz,d{S)] is the (5-entropy of Z. 

The next assumption requires that the mapping C '?^[C] = w he continuous in the supremum 
norm. 



Assumption (A2). For any e > there exists 7 > such that for all (1,(2 £ ^ satisfying 
d(Ci) C2) < £ one has 

sup sup \ wi{t, x) — W2{t, x)\ < 7, 



t X 



where wi{t, x) = (l)[Ci]{t, x), and W2{t, x) = (j)[C2]{t, x). 

Because is linear in w, this assumption along with Assumption (Al) guarantees that all the 
considered objects are measurable. 
Consider the class of densities 



(3.8) 



J" 



p : 



P > 0, p = 1 



< f«, < 00 



00 — ^00 



It is easily seen, that the inequalities of Theorem 1 and Corollary 3 can be made uniform with 
respect to the class T. Indeed, the bound of Corollary 3 remains valid if one replaces ||/||oo and 



by foo and f^^^ ^^'^ respectively. From now on we suppose that / G J^, and, without loss of 
generality, foo > 1- 



3.2.1. Uniform non-random bound. Theorem 1 together with Corollaries 2 and 3 shows that 
Assumption 1 is fulfilled for ||C«;||s,t with g{x) = e~^, 



(3.9) 



U{w) = U.iw,f) := <^ 



SG [1,2) 
s = 2. 



_ ci (s) [ V^S, {w,f) + 2n^/'Ms {w)] , s > 2; 



A^{w) = Al{w) 



37nM^{w), s <2, 

2f^ nM^{w) + Mj{w) s = 2, 

2c3(s)f^ [nM\^{w) + A./^M2{w)M,{w) + SnhM'^^{w)] , s > 2; 



and B{w) = B^{w) 
Put 



|c*(s)Ms(t/;), where c*(s) is defined in (3.2) 



(3.10) 
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Let >cu^{-) be given by (2.7) with U = U^, and 

C7|(y) = 1 + 2^Aa^ + 2yAB^, y > 0, 

where Aa and are defined in (2.8); see also (2.9). 

Theorem 2. Let s > 1 be fixed, (3.7) hold, and let Assumption (A2) be fulfilled. If >i:u^{a) < a 
for all a € [r^, R^] then for any f & J-, y > 1 and e G (0, 1] one has 



E sup 



\ sup [\\U\s,r-u,CI{y)Ui:{w,f)] >0| < -NzA<^/8)logJ^)[l + Li%]e 

1 25(^+i)n" 



\\U\s,r-n,CI{y)U^{wJ)\^< -^-—±T{q + l)N^,,{e/8) [R^CI{1)Y[1 + L^^^je-y/' 
where = 2*^(1 + e), r(-) is the gamma-function, and 

oo 

(3.11) L^l = I^exp [2£j^Ae2-^) - (9/16) 2^k~'']. 



k=l 



The proof follows immediately by application of Corollary 1, and noting that for g{x) = e ^ the 
quantity L^f^ is given by the above formula [cf. (2.13)], while Jg^\-) for g{x) = e~^ is bounded as 
follows 



/oo 
{x - 1)"-^ [e-'^ + dx < T{q + 1) [l + 4%] (2/z)'?e- 



z/2 



Remark 5. It is instructive to compare the results of Theorem 2 with those of Theorem 1 (and 
Corollaries 2 and 3). The uniform bound on \\iw\\s,T in Theorem 2 is determined by the individual 
bound U^{w, f) for a fixed weight w G W , and by the function C|(-) which, in its turn, is computed 
on the basis of A^{w), B^{w) and U^{w,f). The function C|(-) depends on the parametrization 
(3.7) and on the distance d on 3 via the quantities A a and Ab [see (2.8)]. The right hand sides of 
the inequalities in Theorem 2 depend on massiveness of the set of weights W as measured by the 
entropy £z,d{')- Note also that these bounds decrease exponentially in y. 

3.2.2. Uniform random bound. The uniform non-random bounds on ||^.io||s,r given in Theorem 2 
depend on the density / via U^{w, /). As discussed in Introduction, this does not allow to use this 
bound in statistical problems. Our goal is to recover the statement of Theorem 2 (up to some 
numerical constants) with the unknown quantity U^{w,f) replaced by its estimator U^{w). Note 
also that U^{w,f) of Theorem 2 depends on / only if s > 2; here the quantity depending on / is 
^s{w,f). 

Assume that the conditions of Theorem 2 are satisfied, and let s > 2. For any t £ T define 

n 

(3.12) S,(«;) := IIS^II,,,, Sl{t) := -J2w^t,X,), 

^ i=i 

(3.13) U^{w) := Cl{s)\^/7^tsiw) +2n'^/''Ms{w) 
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It is easily seen that U^{w) is a reasonable estimate of U^{w,f) because under mild assumptions 
for any fixed t £ T hj the law of large numbers 



'S'^(^) ~ Olli ly' ~^ ^ ~^ CO, in probability. 



Moreover, 



\T.s{w) -T,s{w,f)\ < \\Sw - \\w{-,-)\\2y\ 



< 



villi/' 



1 9^ 



\wi-,-)\\ly\\s 



Si - \\w{-. 
1 " 



1=1 



Thus, for any s > 2 we have 
(3.14) 



< 



n 



that is, the difference between Tisiw) and T,s{w,f) is controlled in terms of HCju^ ||s/2,t- The idea 
now is to use Theorem 2 in order to find a non-random upper bound on HCtu^ 11^/2 r- can expect 
that this bound will be much smaller than 'Ssiw, f) provided that the function w is small enough. 
If this is true then T,s(w) approximates well '^si'w, /), and it can be used instead of '^si'w, f) in the 
definition of the uniform over W upper bound on ||C«;||s,t- 

In order to control uniformly HCio^ ||s/2,t by applying Theorem 1 and Corollary 1 we need the 
following definitions. Put 



(2,4), 
s > 4; 



sG (2,4), 



s > 4; 



and Biyw"^) := |c^,(s/2)Ms/2(ii'^)5 where c^,(-) is given in (3.2). 

For any subset Z C Z, let >Cfj{Z), A^(Z), and A^(Z) be given by (2.1)-(2.3) with U = U, A = A 
and B = B. With and i?g defined in (3.10), let 



(3.15) 
and we set 



= {C G Z : a/2 < U^{w, f) < a}, a G [r^, R^], 



(3.16) x^(a) := Xj^(Za), = sup a ^A^(Za), = sup a ^A^(Za), 

[cf. (2.7) and (2.8)]. It is important to emphasize here that in the definition of Xjy, A^ and A^ we 
use the same set as in the definition of X[/^(-), Aa^(-) and Kb /.{■)■ 
The next result establishes a random uniform bound on ||^«;||s,t- 
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Theorem 3. Let s > 2 be fixed, (3.7) hold, Assumption (A2) he fulfilled, and 



(3.17) >cu^{a) <a, Va G [r^,/?^]. 

Let e G (0, 1] be fixed, and there exist a positive number 7 < [4ci(s)(l + e)]^^ such that 

(3.18) x^{a) < (7a)^ Va G [r^,R^]. 
Lf Uj denotes the root of the equation 

(3.19) ^A^ + yA^ = 72, 

and if y-y > 1 then: 
(i). For every y G [1,2/7] '^'^^ ^'^^ 



sup {\\U\sr-^e{l)Cl{y)U^{w)y < Ti^q (y) ] %xp{ -y/2} , 



E 



where u^i'y) := Ue[l — 4ci(s)(l + 5)7] ^, anrf = 2^(1 + e). 
(ii). For any subset Wo C W one /las 



E 



sup C/5 (10)1 ' < [l + 4ci(s)(l + e)7]^ sup [U^{w,f)Y 



(3.20) + T2,JV^ sup Msiw)y exp{-y^/2} . 

The explicit expressions for the constants Ti^^ and T2^t oltc given in the beginning of the theorem 
proof. 

Remark 6. 

1. Theorem 3 requires two sets of conditions: conditions of Theorem 2, and conditions on behavior 
of the functions >^^(-); ^a(') '^'^^ ^b(') ^''^ slices Zq defined through U^{w, /). 

2. The parameter 7 controls closeness of U^{-) to U^{-,f): the smaller'^, the closer the random 
bound U^{-) to the non-random one U^{-,f) [see (3.20)]. In this case we do not loose much 
ifU^{-,f) is replaced by its empirical counterpart U^{w). Clearly, it is possible to choose 7 
small and simultaneously to keep large only if and are small enough. Fortunately, 
this is the case in many examples. 

3. Note also that when 7 approaches [4ci(s)(l + e)]~^ the parameter Uei'y) increases to in- 
finity (clearly, we want to keep tie (7) cls close to one as possible). Thus, the assumption 
7 < [4ci(s)(l + e)]""*^ is important; this poses a restriction on the parameter set W. We 
conjecture that the following condition is necessary: for given s > 2 there exists a universal 
constant, say, c{s), such that 7 < c(s). 

The next corollary to Theorem 3 will be useful in what follows. 

Corollary 4. The statements of Theorem 3 remain valid if U^{w,f) and U^{w) are rede- 
fined as ma,x{U(^{w, f), ^/nM2{w)} and max{U^{w), ^/nM2{w)} respectively. 
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3.3. Unifrom bounds for classes of weights depending on the difference of arguments. As we have 
seen, the results and assumptions in Theorems 2 and 3 are stated in terms of the quantities (such 
as A^, A^, y^) that are given imphcitly. In particular, additional computations are still necessary 
in order to apply Theorems 2 and 3 in specific problems. In this section we specialize the results 
of Theorems 2 and 3 for the classes of weights W depending on the difference of arguments. Under 
natural and easily interpretable assumptions on the class of such weights we derive explicit uniform 
bounds on the norms of empirical processes. 

Throughout this section X = 1~ = M"', t = u = mes is the Lebesgue measure, and we write || • \\s 
instead of || • H^^,-. In this section the class of weights W is a set of functions from x R"' to M of 
the following form 

(3.21) W = {w{t-x), w G V}, 

where V is a given set d-variate functions. For the sake of notational convenience we will identify 
the weight w £ W with the d-variate function w £ V in the definition of the process S^y^ and the 
quantities such as U^, A^, etc. Thus when we write li; G W we mean the weight w{- — ■) while 
w £V denotes the corresponding d-variate function; this should not lead to a confusion. 

Let (3, d) be a fixed metric space; as before, we suppose that V is parametrized by the parameter 
C G 3, i.e., 

(3.22) V = {w:w = (/>[C], C G 

where Z is a totally bounded subset of the metric space (3,d). Recall that A^z,d('^)) 5 > is the 
number of the balls of the radius 5 in the metric d that form a minimal covering of the set Z. 

We need the following assumptions on the class of weights W (the functional set V). 
Assumption (W). 

(Wl) The Lebesgue measure of support of all functions from V is finite, i.e., 
(3.23) /U* := sup mes{supp(w)} < oo. 

to6V 

(W2) There exist real numbers a\ G (0, 1) and Q2 G (0, 1) such that 

mes|x G M*^ : |w(x)| > ai||tf;||oo| > a2 nies{supp(w)}, Vw G V. 
(W3) There exists a real number > 1 such that 

nmes{supp(?i;)} > fi, Vu; G V. 
(W4) There exists a real number (3 £ (0, 1) such that 

sup I In [iVz,d('5)] - =: CM < oo. 

<56{0,1) 

Remark 7. We will show that Assumption (W2) is fulfilled ifV is a set of smooth functions. 
Assumption (W3) together with (W2) allows to establish relations between 'hp-norms of functions 
from V; this will be extensively used in what follows. Assumption (W4) is a usual entropy condition. 
In particular, (W4) ensures that the quantity L^xp in (3.11) is finite. 

15 



In addition to Assumption (W) we will need the following assumption on the properties of the 
mapping (j) in (3.22). For p > 1 put 



(3.24) 

and define 
(3.25) 



< w„ := n^/P inf \\w\L < n^l^ 



sup lltfllp =: Wp < oo. 



%(h):={C^'L:n^l^\\m\\v<^]^ ^ e t 



Wp,WpJ. 



Assumption (L). The mapping (j) in (3.22) satisfies the following conditions: 
• if s £ [1, 2) then 



sup 



d(Ci,C2) 



• if s >2 then 
(3.26) 



sup 

<i,C2eZ2(fe) 



V^U[Ci]-cl^[C2]' 
d(Ci,C2) 



< b, V6 G [w2, W2]. 



We note that Assumption (L) guarantees continuity of "^[^l for any s < 2. The same 

property for s > 2 follows from Lemma 7. This, in view of Remark 2, replaces Assumption (A2). 

The next statement presents the uniform moment bound on H^^roHs when s E [1,2], and W is 
given by (3.21). 

Theorem 4. Let the class of weights W be defined by (3.21), and let (3.22) and Assump- 
tions (Wl), (W4) and (L) hold. 

(i). If s e [1, 2) then for alln>l,z> [\/37/2] and e G (0, 1] one has 

.2 



E sup 



4:uJl + z)n^/'\\w\\s < Ts.n'i/ 



3.n^' exp 



2z^ 
~37 



-n 



(2/s)-l 



(ii). If f € then for all n> 1, z > A/8[/i*f^ + 4n ^/^J, and e G (0, 1] one has 



E sup 



Uwh-U,(l + Z + z'^/12^^/^\\w\\2y 



< r4,, n''/^ exp 



16[f23^, +4n-i 



/2l 



The explicit expressions for the constants T^^^ and T/^^;: are given in the beginning of the proof of the 
theorem. 

The bound of Theorem 4 is non-random because U^{w,f) does not depend on / whenever 
s G [1, 2]. The proof of this statement is based on application of Theorem 2. 

Now we proceed with the case s > 2. Here we need the following notation. Given p > 2, let 
rup G (0, 1] be such that 



(3.27) 



,_i ^'/1k[Ci]-0[C2]|L ^ 

sup b sup =: Cp < 00. 

be[w2,w2] Ci,C2eZ2(6) [aiuA2jJ " 
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Existence of nip G (0, 1] such that (3.27) holds is ensured by Lemma 7 given in Section 8.2. In 
particular, it is shown there that if Assumptions (W) and (L) hold then nip can be taken equal to 
2/ p. We note also that ni2 = 1 and C2 = 1 by Assumption (L). 
Following (3.12)-(3.13) and Corollary 4 we set 



ci{s){ ^/n 



1 " 



1/2 



+ 2n 



(3.28) 
Put also 



lJ^{w) := max{tj^{w) , \/n\\'w\\2} , U(^{w) := mayi{U^{w, f),\/ri\\w\\2}- 
(3.29) Cliy) = l + 2^o{Vy[f''J' + n-'/^'''^] +yn-'/^], 



m :- 



1 A nis, s £ (2,4) 

1 A mg A s > 4, 



1 — 1 /2 

where i^o •= 5ci(s)[Cs V l]fooa|f 02 , ai and 02 are given in Assumption (W2), and nip and Cp 
are defined in (3.27). 

Theorem 5. Let Assumptions (W) and (L) hold, and assume that f £ T. Suppose that (W3) 
is fulfilled with j^i > [64cf (s)] sa4-i ^ and (W4) is fulfilled with jS < ni. Let and 



(3.30) 



y* 



with constants i?i and i?2 specified explicitly in the proof; then for any s > 2 and y € one has 



se (2,4), 
s > 4, 



E 



sup \\\U\s-^eh)Cliy)U^{w)y < n,,n'^/^[qiy)Yexp{-y/2}, 



where Me(') is defined in Theorem 3. In addition, ifWo C W is an arbitrary subset ofW then 



E 







sup U^{w) 


"<-{ 







l + 4ci(s)(l + e)^ 



_L 1_ 

sA4 2 



9(^-2) 



sup C/5(t(^) +TQ^en 2s exp{-y*/2}. 



The explicit expressions for the constants T^^^ and Tg.e are given in the proof. 

Theorem 5 establishes random uniform bounds on the norms of empirical processes in terms of 
the parameters determining the class W. In particular, the parameters ^ and fi^, play an important 
role. Theorem 5 leads to a number of powerful asymptotic results that demonstrate sharpness of 
the proposed random bound. 

Corollary 5. Let assumptions of Theorem 5 hold, and let s > 2 be fixed. There exist positive 
constants ki = ki{s), i = 1, 2, 3 such that if 



fi = fin^ [Inn] 
then for all i > and q > 1 



lim sup n E sup 



IJ-* = IJ'*,n ^ i^un] ^, e = e„x[lnn] n — > 00, 
- (1 + 3en)U^{w) 



0, 



E 



sup U^{w) < (l + e„) sup U^iwJ) +Rn{Wo), 



w/iere limsupn^oosupjgjrsupyypcw [n^Rn(Wo)] = 0. 
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The explicit expressions for the constants /cj > 0, i = 1, 2, 3 are easily computed from Theorem 5. 

Remark 8. Corollary 5 shows that if the class of weights W is such that n = fi^ and /i^, = /U^,.„, 
and if e is set to be e = e„, then (1 + 3e„)?7„(w) is a uniform random bound on \\^w\\s which 
is asymptotically almost as good as the non-random bound U^{w,f) depending on f. Thus, in 
asymptotic terms, there is no loss in sharpness of the random uniform bound in comparison with 
the non-random bound that depends on f . 

3.4. Specific problems. In this section we consider process corresponding to special classes 
of weights W that arise in kernel density estimation. Using results of Theorems 4 and 5 we derive 
uniform bounds on the norms of these processes. As in Section 3.3, here X = T = M*^, and v and 
T are both the Lebesgue measure. 

Let /C be a given set of real functions defined on and suppose that /C is a totally bounded set 
with respect to the Loo~norm. Letn:= <S)f^^[hf"\ hf where the vectors /i™" = (/if™, . . . , h^'""), 

/jinax ^ J/^max^ _ _ _ ^ /^max^^ g < /l™" < /if^^ < 1, Vi = 1, . . . , d are fixed. 

For any h £ Ti define := nf=i ^'^d endow the set Ti with the following distance: 

(3.31) An(h,h')= max In f ^^i^ 

In order to verify that A-^ is indeed a distance on T-L it suffices to note that the function (x, y) i— ?■ 
ln(x V y) — ln(x A y), x > 0,y > satisfies all the axioms of a distance on M+ \ {0}. 

We will be interested in the following classes of weights W and the corresponding processes 

Kernel density estimator process. With any K £ fC and h £ H we associate the weight function 



w 



(t-x) =n-^Kh{t-x) := {nVhy^K[{t - x)/h]. 



As before, u/v, u,v £ M.'^, stands for the coordinate-wise division {ui/vi, . . . ,Ud/vd). 
The weight w is naturally parametrized by K and h so that we put 

(3.32) Z^^'> := JC X n, C = {K,h), w = (t)i[C,] := n'^ Kj,. 

We equip Z^-^^ with the family of distances {d^^''(-, •),'d > 0} defined by 

d^')(C,0=^max{||A'-i^'||^, An{h,h')}, C = {K,h), C'={K',h'), i) > 0. 

Obviously, Z^^^ is a totally bounded set with respect to d^^^ for any > 0. 
The corresponding family of random fields is 



1 

(3.33) d'^(t):=e<^,[^](t) = -J]{K/,(t-X,)-EK4t-X)}, ( £ Z^'^ 

i=l 



and we are interested in bounds on the L^-norm of this process uniform over the class of weights 

W^^) := [w{- -■) = n-^Kh{- - •) : {K,h) £ Z^}. 
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We note that ^i; is the stochastic error of the kernel density estimator associated with the kernel 
K £ IC and bandwidth h € T-L. According to Theorems 4 and 5, for the process G the 

uniform bounds on should be based on the following functionals. Define 

I 

For s > 2 we put 

U^^'\w,f) := ci( 

?7«H ■.= c^{s) 
and finally 

(3.34) ^ 

Convolution kernel density estimator process. For any {K, h) G Z^^^ and {Q, f)) G Z^-*^) we define 

(3.35) t/;(t-3;) =n-i[i^/,*Qf,](t-x), 

where Z^^) is defined in (3.32), and * stands for the convolution on M.'^. Put 

Z(2) := Z« X Z(i\ z=[{K,h),{Q,t))], w = M^] = n-\Kh * Qt,), 
and define the family of distances on Z^^-* as 

df (z,z') = ^9max{||K-i^'|U V||Q-Q'||oo, A^(/i, /i') V A^(f,, [)')}, > 0, 
where z = [{K, h), (Q, f))] , z' = [{K', h'), {Q', t)')] , z,z' € lP-\ Obviously, iP^ is a totally bounded 

(2) 

set with respect to the distance d^ for any i? > 0. 
The corresponding family of random fields is 

1 

(3.36) (i) := (*) = - E { [^'^ * ] - ^0 - ^ [^^ * Qf,] (t - X) } , C G ^^'^ , 

i=l 

1 1 (2") 1 1 

and we are interested in a uniform bound on over 

>V(2) :=|^(.-.)=n-iK,,*Qf,(.-.), [(K,/i),(Q,f))] GZ(2)}. 

The random field with w given by (3.35) appears in the context of multivariate density estima- 
tion. In particular, the uniform bounds on ||i^«,||s are instrumental in construction of a selection rule 



4(nh)i/--i||K|U SG[1,2), 
(n/i)-V2 ||i^||2, s = 2. 



n 



-1/2 



n 



-1/2 



-1 s/2 \ 

i^|(t- j;)/(x)dx dt +2(n/i)^/'~^||i^| 



s/2 



j=i 



^^Kl{t-Xi) dt +2(n/i)i/^-i||K 



1/. 



max [?7^^"^''(ii;, /), (n/i) 

:[{7f )(«;), (n/i)-V2||i^||2] 



max 
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for the family of kernel estimators parametrized hjICxTi [see Goldenshluger and Lepski (2009)]. 
Theorems 4 and 5 suggest to base the uniform bounds on the following quantities. Define 

Any'-^Kh*Qt,\\s, SG[1,2), 
n-y'^\\Kh*Qt,\\2, s = 2. 



UPiw) 



For s > 2 we put 



n 



-1/2 



[Kh*Qi,]\t-x)fix)dx 



s/2 \ 1/s 



and finally 
(3.37) 



ci(s) 



n 



-1/2 



n 



i=l 



[Kh*Qi,]\t-Xi 



s/2 



dt 



dt 

l/s 



+ 2n'/'~'\\Kh*Qi,\U 
+ 2n'/'"'\\Kh*Qt,\\ 



ijf\w) 



max [[/f )(«;,/), n-V2||K^*Q^||2], 
max [?7P^(i(j), n~2\\Kh* Q^\\2\. 



Theorems 4 and 5 can be used in order to establish upper bounds on the norms of the processes 



1, 2. For this purpose, Assumptions (W) and (L) should be verified for the classes of weights 



W(*)^ i = 1,2 defined above. To this end we introduce conditions on the family of kernels K, that 
imply Assumptions (W) and (L). These conditions are rather natural and easily verifiable; they can 
be weakened in several ways, but we do not pursue this issue here and try to minimize cumbersome 
calculations to be done. 

Assumption (K) . 

(Kl) The family K, is a subset of the isotropic Holder hall of functions ^di^-, L/c) with the exponent 



1 and the Lipschitz constant Lfc, i.e., 

\K{x)-K{y)\<L!c\x-y\ 



\/x,y G 



where \ ■ \ denotes the Euclidean distance. Moreover, any function K from K, is compactly 
supported and, without loss of generality, supp(K) C [—1/2, 1/2]^^ for all K £ IC. 
(K2) There exist real numbers ki > and koo < oo such that 



kl < 



K{t)dt 



< 



|-^||oo ^ koo. 



Without loss of generality we will assume that koo ^ 1 cind ki < 1. 
(K3) The set K, is a totally hounded set with respect to the hao-norm, and there exists a real numher 
PfC £ (0, 1) such that the entropy £-k.{') of IC satisfies 

sup [£k{Si) - =: Cfc < oo. 

<5e(o,i) 

Several remarks on the above assumptions are in order. First we note that Assumptions (Kl) and 
(K3) are not completely independent. In fact, if we suppose that /C C M(i[a,L;c) with some a > d 
then Assumption (K3) is automatically fulfilled with /3ic = a/d. On the other hand, all our results 
remain valid if /C C Mdi^a, Ljc) with some a > 0. Observe also that the condition | J K{t)dt\ > ki 
of Assumption (K2) is not restrictive at all because for kernel estimators J K(t)dt = 1. Therefore 
the first inequality in (K2) is satisfied with ki = 1. 
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Remark 9. It is easy to check that Assumption (Kl) implies Assumption (A2) in Section 3.2 
and Assumption 2 in Section 2. 



Now we apply Theorems 4 and 5 to the famihes of random fields given by (3.33) and (3.36). We 
present the results for the processes {C0i[c])C ^ ^'■^•'} and {i^^iz]-!^ ^ Z^^)} in a unified way. 

3.4.1. Case s E [1, 2] . Uniform non-random bounds. In order to derive the uniform upper bounds 
for s G [1, 2] we use Theorem 4. Obviously, Assumption (K) implies Assumptions (Wl) and (W4). 
Thus in order to apply Theorem 4 we need to verify Assumption (L). This is done in Lemma 9 
given in Section 9. Thus Theorem 4 is directly applicable, and non-asymptotic bounds can be 
straightforwardly derived from this theorem; one needs only to recalculate the constants appearing 
in the statements of the theorem. 

We note that the quantity /i* defined in (3.23) satisfies /x* < 2'^V/imax for the set of weig hts W(^) 
and /i* < 4'^V/imax for the set of weights W^^^ If we assume that V/^max — as n — t- oo then we can 
establish some asymptotic results, one of which is given in the next theorem. 

Theorem 6. // Assumption (K) holds then for all s £ [1,2), £ > and e G (0, 1) 



lim n sup E sup 



lie 



(l + e)[/f (w; 



0, i = 1,2. 



If Assumption (K) holds and V/jn 



0(1/ In n) as n —)• 00 then for all i > and e G (0,1) 



lim n sup E sup 



0, i = l,2. 



Proof of the theorem is omitted; it is a straightforward consequence of Theorem 4 and Lemma 9 
given below in Section 9. 

3.4.2. Case s > 2. Uniform random bounds. In the case s > 2 the uniform bounds are derived 
from Theorem 5. To state these results we need the following notation. Define 



(3.38) 



i^^Q^ := Wci{s)iJLicVd /kl 



d/2 



^^^^ := 10ci(s)foo[2^+'VdLyckoo/k? 



d/2 



The next two quantities, A-^ and -B-^, are completely determined by the bandwidth set T-L: 

d d 

' l„« f },max /f,min^ 



(3.39) An := n 1" [hT/hf'') , := logs (l4n.ax/14^,„ j = J] logs [hf^Vhf 



For y > put 

(3.40) C|,(y):=l + 2.?«{Vy([: 



2 ' 



1/. 



+ n 



-l/2s 



yn 



Define also 



y* 



4*^(nyftmi„)-V2 r(2rfil4max)2/^ + tT^I' 



SG (2,4), 
, s>4. 



1,2. 



where explicit expressions for the constants tJ^^^i^s^ 
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1, 2 are given in the proof of Theorem 7. 



Theorem 7. Let Assumption (K) hold, f ^ F, and let rcLa.y.j=i^,,,^d\hf^^\ < 1. For i = 1,2 
assume that 



(3.41) 



nVf^min > 



d{i-l) 



If J := (nV^min)"'^/^*^'^-' then for any s > 2, y £ [l,yi*''] and for i = 1,2 one has 



E sup 



{ - n,{^)ClMuf < fl'}{l + Anr{l + Bnh''/'[Cl{y)Ye 



ip-y/^ 



where Ue{-) is defined in Theorem 3, and tj^\w) are defined in (3.34) '^'^d, (3.37). 
In addition, for any subset Wo ^ W^*^, any s > 2 and for i = 1,2 one has 

1 + 4ci(s)(l + e) fjiV/jmrnV^" ^ sup 
^ ^ -I weWo 



E 


sup tjc'\'w) 


< 




L w&Wo ^ 





f« (1 + Anf'a + Bn)n'^ exp{ - yi'> ft}. 



The explicit expressions for the constants T^fl and T2I are given in the proof. 

(i) 

We emphasize that the upper bounds of Theorem 7 are non-asymptotic. The constants t?^ , 
jtW g^j^g written down exphcitly in the proof of the theorem; they are completely 
determined through the quantities L/c, ki, koo, Cic and (3ic appearing in Assumption (K), and the 
constant ci{s) in the Rosenthal inequality. 

Remark 10. The condition (3.41) is not restrictive because the standard assumption on the 
bandwidth set % in the kernel density estimation is that 

nVj^min — )• 00, Vftmax — )• 0, 72 — )• OO . 

The bounds established in Theorem 7 can be used in order to derive asymptotic (as n — t- oo) 
results under general assumptions on the set of bandwidths % . One of such results is given in the 
next corollary. 

Corollary 6. Let s > 2 be fixed, Assumption (K) hold, and f £ J-. There exist positive 
constants ki^i = A;i.i(s), k2.i = k2.i{s), and k^^i = k^^i^s), i = 1,2 such that if 



In 71 



, nV^min 



In 71 



[Inn] n 



oo. 



then for all ^ > 0, ( 


7> 1 














lim sup E 


sup 


■||e«||^-(l + 36„)?7f 




g 

= 0. 

+ 


In addition, for any subset Wq € 


V\;{«) 


one has 






E 


sup ui^\w) 

- w&Wo 


< 


(l + e„) sup U^'\w,f) 


' + i?«(Wo 



where limsup^^^^ supjgj- supy^^^cww [^^-^"^^o)] = 0, z = 1,2. 

We remark that explicit expressions for the constants ki^i and /c2,«i i = 1,2 are easily derived 
from Theorem 7. 
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4. Uniform bounds for norms of regression type processes. In this section we use 
Proposition 2 in order to derive uniform bounds for the family ||77«,||s,r! w E W. First we check 
Assumption 1 by establishing an analogue of Theorem 1 for a fixed weight function w G W [see 
Theorem 8 below]. It turns out that the corresponding inequality depends heavily on the tail 
probability of random variable e. In other words, we prove that Assumption 1 is fulfilled with a 
function g that is determined by the rate at which the tail probability of e decreases. Next, under 
Assumptions (W) and (L), we derive uniform bounds using Corollary 1; this leads to an analogue 
of Theorem 4 for the regression-type processes. 

4.1. Probability bounds for fixed weight function. We consider two types of moment conditions 
on the distribution of e. 

Assumption (E). The distribution of e is symmetric, and one of the following two conditions 
is fulfilled 

(El) there exist constants a > 0, f > and b > such that 

F{|e| > x) < uexpf-fex"}, Vx > 0, 
(E2) there exist constants p > [s V 2] and P > such that 

EkP < p. 



Let a"? := Ke^ and e. 



QsiwJ) 



(E|e|'^)^/'*. For any w € W define 

^ ci(s)[a,V^S,(?i;,/) + 2nV«e,M,( 



w) 



s < 2, 
s = 2, 
s>2, 



M2(u;)[(6cj2 +8)n + 96(7<,nV«], s < 2, 
Ga^nMl^y (w) + 24ae^/T^M^{w) s = 2, 

and if s > 2 then we set 

vj'iiwj) := 6c3{s)\alnM\^ iw) + 4a,V^i:s{wJ)Msiw) + 8esn^/'M^{w) 

In the above formulas we use notation introduced in the beginning of Section 3; the formulas should 
be compared with (3.3) and (3.4). 

The next theorem is the analogue of Theorem 1 for the regression-type processes. 

Theorem 8. (i). Suppose that Assumption (El) holds, and for x > define the function 

exp{ - A ^ < 2, 



(4.1) Gi{x) := {1 + nv)ga,b{x), ga,b{x) : = 
Then for all s G [1, oo) and z > one has 

^{\\r]w\Ur > 0s{wj)+z} <Gi(^j 
where c*(-) is given in (3.2). 
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exp 



{ - |x| A |6i/"x|"/(i+°)}, s>2. 



w1{wJ) + \c,{s)Ms{w)z 



(ii). Suppose that Assumption (E2) holds and for x > define the function 



G2[x) = {l + nP) X 
Then for all s G [1, oo) and z > one has 

^{\\Vw\\s,r>Qs{w)+z}<G2 



x~^phi [l +p~^x'\Y ^ 



s <2, 
s>2. 



4.2. Uniform bound. Theorem 8 guarantees that Assumption 1 holds with function g being 
either Gi or G2. This result is the basis for derivation of uniform bounds, and the general machinery 
presented in the previous sections can be fully applied here. In this section we restrict ourselves 
only with uniform bounds over the classes of weights depending on the difference of arguments. In 
other words, under Assumptions (W), (L) and (El) we prove an analogue of Theorem 4 for the 
regression-type processes. 

A natural assumption in the regression model where the process {r]iu,w G W} appears is that 
the design variable X is distributed on a bounded interval of W^, i.e., the density / is compactly 
supported. This will be assumed throughtout this section. 

Let I £ he a bounded interval, T = X = I, and let r = = mes be the Lebesgue measure. 

— 1 —1/2 

For the sake of brevity we write a* = a, ar, 
Define 



a := max \/mes{I), ci(s) [o"ef(^^ + 26^0 
[2a2 + f + 32^,nV-i];,2/.-i^ 
2f^/i, + 8n-V2 
I 2c3(s)f^ [a^,^il/' + (4cT,a. + Se.a^),^-!/^ 



1^02 '''^ where ai and 02 appear in Assumption (W2). 

4 



3 

s < 2, 
s = 2, 
s > 2. 



Theorem 9. Let Assumption (W) holds, f £ J- , and assume that (3.26) is valid for all s > 1. 
Let Assumption (W5) he fulfilled with f3 < a/{2 + a), if s < 2, and with /? < a/(l + a) if s > 2. 
Then for all s > 1, q > 1 and y > 1 one has 



E sup 



[l + 2^b„ + 2yCn)Vn\\w\\2 < r„„[l + 2^b„ + 2yCnY [gaMiv)] 



1/4 



where = 2'^(1 + e), ga,b{') is defined in (4-1)' ^''^^ the explicit expression of the constant T^^e is 
given in the beginning of the proof of the theorem. 

The following asymptotic result is an immediate consequence of Theorem 9. 

Corollary 7. Let assumptions of Theorem 9 hold. For any a > there exist a universal 
constant c = c(a) > such that if fi^ x [Inn]"'^ then for all s > 1, e £ (0, 1) and for all £ > 



lim n supE sup 



[1 + e)a^||u;||2 



0. 



The explicit expression for c(q) is easily derived from Theorem 9. 
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5. Proofs of Propositions 1 and 2. 

5.1. Proof of Proposition 1. Let Z^, /c G N be an e2~'^~'^-net of Z, and let Zk{C)j C ^ ^ denote 
the element of Z^ closest to ^ (in the metric d) . 

The continuity of the mapping ( i— )■ (^^p^Q guarantees that P-almost surely the following relation 
holds for any C G Z: 



k=0 



where (^'^^ is an arbitrary fixed element of Z and zq{(^) = C^^\ VC G Z. 
Note also that independently of (" for all A; > 

(5.2) d(zfc+i(C),Zfc(C)) <e2-^'"'- 

We get from sub-additivity of ^, (5.1) and (5.2) that for any C ^ 

9 OO 

6 



k=0 



2 



(5.3) < <f + — sup sup {k + iy^ (^^[,] - , 

d{z,z')< €2-'=-2 

where pk := 6/{n'^{k + 1)^) and Efclo^'fc = 1- Since ^, is linear, ^^j^j - ^.^[^/j = C<^[2]_0[^'] for all 
z, z' G Z, and we obtain from (5.3) and the triangle inequality for probabilities that 

P{sup^(%]) > (l + e)[xc;(Z) + C*(2/,Z)]} < p{M'(e^[^(o)]) >x^(Z) + C*(2/,Z)} 



(5.4) 



k=0 (z,z')eZk+iXZk: ^ \ I ) 

d(z,z')< e2-'=-2 



In view of (2.1) and because C(°) G Z we have that [/(^[C^"^]) < xij{Z). Therefore, we get from 
Assumption l(i) and monotonicity of the function g that for any ?/ > 



h < P{^(e^[^(o)]) -^(</.[C(°)]) >C*(y,Z)} 



/ [C*{y,Z)Y \ f [C*{y,Z)Y \ 

^^■^^ - {<Pm)+B{^m)C*iy,Z)) - \Al{Z) + AB{Z)C*{y,Z)) " ^^^^^ 

To order to get the last inequality we have used monotonicity of g and that for any y > 
[C*{y,Z)]' _ [^AA{Z)+yAB{Z)]^ 



A\[Z) + AB(Z)C*(y, Z) A\{Z) + Ab{Z)[^Aa{Z) + yAsiZ 
By (2.1), if z,z' eZ and d(z, z') < e2-'=-2 then 

U{,p[z]-(b[z']) <e2-''-^MZ), 
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and, therefore, for any y > 

Qe[Ku{Z) + C*{y,Z)\ 



> 



7r2(fc + l)2 



< ^^^{i4>[z]-4>W])-U{M-<t>[z']) > 



16(fc + 1)2 



Here we took into account that minfc>o [Gtt ^(/c + 1) 2 _ 2 ^] > 0, and 9/16 < (6/7r2). Putting 
~ ^I6{k+i)'^ s-PPlyiiig Assumption l(i) we obtain for any z,z' G Z^+i x Zj. satisfying 

diz,z') < e2-'=-2: 



6e[xu{Z) + C*{y,Z) 



< 9 



7r2(A; + l)2 
a 

k 



cl 



{ci>[z\-m)+B{m-m)Ck 



<9[- — , , . ——]<9( 



[Aa (Z) 62-^-2] 2 + ^2-'=-2] V {Z) + Ab (Z) (7j ' 

where we denoted Ck = Ck2^'^^. Taking into account that 9{k + 1)^22^-2 > k > 0, and 

by definition of C*{y, Z) we obtain for any y > that 

> 9y(fc + l)-22'=-2. 



Al (Z) + Ab (Z) Ck 
Hence, we obtain for any z,z' G ^fc+i x Z^ satisfying d{z,z') < e2^^~^: 

(5.6) P{*fow-,M) > ^ + 1)-'). 
Noting that the right hand side of (5.6) does not depend on z, z' we get 

(5.7) h < J]{A^z,d(e2-^-i)} g[9y2^-\k + ir 



k=0 

The theorem statement follows now from (5.4), (5.5) and (5.7). 



5.2. Proof of Proposition 2. Let Z;, / = 1, . . . , Ni^^^e/S) be d-balls of the radius e/8 forming 
a minimal covering of the set Z. For any < j < e""*^ log2 {R/t) — 1 (without loss of generality we 
assume that log2(i?/r) is integer), let 5j = r2''^ ^ and put 

= {CGZ:5, <C/(<^[C]) <<5,-+i}. 

Note that Z^^, C Z^^. for all j because e G (0, 1]; recall that is defined in (2.6). 
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We have Zi = uj°!g ^^''''^ ^{ZiH ts^^^ } for any / = 1, . . . , iVz,d(e/8). Therefore for any y > 



(5.8) 



*:^(y,ZO< sup 

j=0,...,log2 {R/r)-l 



sup ^{i^^i^^ -u^C*{y)5j 



Let < j < e ^ log2 (R/r) — 1 be fixed; then usmg the definition of Aa and [see (2.2)-(2.3)] 
and the fact that Za . , , C Za . , , we have that 



'j+i' 



Therefore 



C*iy) > l + dJ^,[2^AAiZs,+,) + 2yAB 

> 1 + [^Aa{Zs^^,) + yAsiZs^^,)] 

> l + dj'[^AA{Zs^^,)+yAB{Zs^+,)]. 

C*{y)6, > 6, + [^AA{ts^+,)+yABm^,)] > 2~'xu{^5,+,) + C*{y,Zs^^,), 



since by the premise of the proposition Sj = 2~^Sj^i > 2~''M:u{'Esj+i) ^ 2~"^X(/(Z5^,^^) . Note also 
that the definition of C*(-,-) implies that C*(-,Zi) < C*{^-,Z2) whenever Zi C Z2. Thus we have 
for any < j < log2 (R/r) — 1 and any 1 = 1,..., Nz,d{^/8) 

(5.9) n,C*{y)6j > {1 + e)[><u{Zi n Zs^^,) + C* {y, Zi n Zs^^,] 

Taking into account (5.8) we obtain 
P{^:^(y,ZO>0} 

log2(iJ/r)-l 

xu{Zi n Zs^^,) + C*{y, Zi n '^s,+,, 



j=0 



sup ^'(e</,[c]) > + 



Applying Proposition 1 for the sets Zi n ^5,^1 we get for any y > 



log2 (/?/r)-l 



F{KAy,Zi)>o} < E ^, 



(^) 



j=0 



"i+i 



log2 {R/r)-l 



It remains to note that the right hand side of the last inequality does not depend on thus we 
come to the first assertion of the proposition. 

Now we derive the bound for the moments of ^'*^(y, Z). We have from (5.8) with y > that for 
any q > 1 

E(supU{^^[^])-u,C*iy)U{m} 

Afz,d(<:/8) e-ilog2{i?/r)-l 



< 



l,i{e/8) e-Hog^ (R/r)~l 

J2 E sup {^(^,[c])-n,C*(2/)5,} 
1=1 j=o ^ c&ZinZs^^^ ' y - 



(5.10) 



Afz,d(e/8) £-ilog2{fl/r)-l 
1=1 j=0 

27 



For / = 1, . . . , A^z,d(e/8) and < j < e"^ logs {R/r) - 1 we have 

Ej{l) = qT [x-u,C*iy)5jy-^P \ sup ^(^^[f]) > xjdx 

= [u,C*iy)Y5]q Tiz-iy-'Fl sup > zn,C*(y)5Adz 

< [u,C*{y)Y5]q r{z-iy-'F\ sup > u,C*iyz)5j\dz 

(5.11) < KC*(y)]''5|g^"(z-ir"i4^)(yz,r2^(^+i))dz. 

Here the third hne follows from zC*{y) > C*{yz) for any z > 1, and the last line is a consequence 
of (5.9) and the probability bound established above. 

The second statement of the theorem follows now from (5.10) and (5.11) since the right hand 
side in (5.11) does not depend on I. | 



6. Proof of Theorem 1. 

6.1. Preliminaries. For convenience in this section we present some well-known results that will 
be repeatedly used in the proofs. 

Empirical processes. Let be a countable set of functions / : A" — )• M. Suppose that E/(X) = 0, 
: < b, y f € T and put 

n 

y = supV/(Xi), = supE[f{X)f. 



Lemma 1. For any x >0 

F{Y -EY >x} < exp 



x^ 



2710-2 + ibEY + |6x j ■ 



The statement of the lemma is an immediate consequence of the the Bennett inequality for em- 
pirical processes [see Bousquet (2002)] and the standard arguments allowing to derive the Bernstein 
inequality from the Bennett inequality. 

Inequalities for sums of independent random variables. We recall the well-known Rosenthal and 
Bahr-Esseen [see von Bahr and Esseen (1965)] bounds on the moments of sums of independent 
random variables. 

Lemma 2. Let Yi, . . . , y„ be independent random variables, EYi = 0, i = 1, . . . ,n. Then 

p/2^ 



e\y^y,^ <[c,{p)f\ Y,ny^? + (Y.^yA 
i=i I i=i ^ 1=1 ^ 



>, p>2- 



n 

IP 



^Y.^ <25^E|y,|P, pG[l,2), 

i=l i=l 
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where ci{p) = Ibp/lnp. 

The constant ci{p) = 15p/lnp in the Rosenthal inequahty is obtained by symmetrization of the 
inequahty of Theorem 4.1 in Johnson et al. (1985). 

Norms of integral operators. In the next statement presents inequahties for norms of integral 
operators. 

Lemma 3. Let (T, T, r) and (^X,X,x) cr-finite spaces, w he a y. X) -measurable function 
on T X X , and let 

Mp^r,xi'w) ■= snp\\w{-,x)\\p^r V sup 

If R G I^p{X,x) andTii{t) := J w{t,x)R{x)x{'ix) then the following statements hold. 

(a) . For any p £ [1, oo] 

(6.1) \\Mp,r ^ ^l,r,xMPIIp,X- 

(b) . For any 1 < p < r < oo 

(6.2) \\Ir\1^^ < C2{p)M,,r,xiw)\\R\\p,^, 

where | = 1 + ^ ~ |) o.'iT'd C2{p) is a numerical constant independent of w. 

The statements of the lemma can be found in (Folland 1999, Theorems 6.18 and 6.36). 

Note that if x = i^' := then Mp^r,x{w) = Mp{w), Vu; [see (3.1)]. If T = X = R'^, t and x 
are the Lebesgue measures, and if w(t, x) depends on the difference t — x only, then C2{p) = 1, and 
(6.2) is the well-known Young inequality. 

6.2. Proof of Theorem 1. We begin with two technical lemmas; their proofs are given in Ap- 
pendix. 

Lemma 4. Let M_s_ he the unit hall in L^(7', r), and suppose that Assumption (Al) hold. 
Then, there exists a countable set £ C such that 

' s-l 

Lemma 5. Let w{t, x) = w{t, x) — Mw{t, X); then for all p > 1 one has 

(a) . \\iv{-,x)\\p^^ < 2sup^.g;^. ||u'(-,a;)||p_^. 

(b) . Mp{w) < 2Mp{w). 

We break the proof of Theorem 1 into several steps. 
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Step 1: reduction to empirical process. We obtain from Lemma 4 



sup / l{t)i^{t)T{dt) 
n 



sup y l{t)w{t,Xi)T{dt) = sup} X{Xi), 



where 

Thus, 
(6.3) 



X{x)= l{t)w{t,x)T{dt), leH 



m,^, = sup^X{Xi)=:Y, 



A6A 



i=l 



and, obviously, EX{X) = 0. The idea now is to apply Lemma 1 to the random variable Y. 

Step 2: some upper bounds. In order to apply Lemma 1 we need to bound from above the following 
quantities: (i) KY; (ii) b := sup^^g^^^ ll^lloo! and (iii) o"^ := sup;^^^ EA^(X). 

(i). Upper bound for EY . Applying the Holder inequality we get from (6.3) 



E 



sup 
AeA 



Elk. 



E\C^{t)\'T{dt) 



If s G [1,2] then for all t e T 

s_ 

E|e^(t)|' < E|e^(t)|' ' < [nEw\t,X)] 



n / w'^{t,x)f{x)i'{dx) 



Thus we have for all s G [1,2] 



(6.4) 



EY = E 



sup^ A(Xj 



AeA 



< ^/nSs(w,/). 



Note that the same quantity can be bounded from above in a different way. Indeed, in view of 
the Barh-Esseen inequality (the second statement of Lemma 6.8) 

E|C,«(t)|'' < 2nE \w{t,X)\' = 2^+'nE \w{t,X)\'. 

and we obtain for all s G [1, 2] 



.5) 



Ey = E 



sup 

AeA 



1=1 



We get finally from (6.4) and (6.5) 
(6.6) E 



Y < {V^5],(«;,/)} A {4ni/^M,(u;)}. 
30 



If s = 2 we obtain a bound independent of /: indeed, in this case 



(6.7) Ey = E 



supV] A(Xi 

A6A 



1=1 



w^(t,x)/(x)z/(dx)r(dt) 



w 



If s > 2 then applying the Rosenthal inequality (the first assertion of Lemma 6.8) to £,w{t), which 
is a sum of i.i.d random variables for any t £ T, we get 



[E|ea,(t)|')] ^ <ci(s) {nEw'^{t,X))'^ +nE\wit,X)\' 



and, therefore. 



E 



sup 
AeA 



i=l 



< ci(s)< ^/n 



w^{t,x)f{x)i^{dx)] r(dt) 



+2n 



l/s 



\w{t,x)\'f{x)i^idx)T{dt) 



To get the last inequality we have used that E\iu{t,X)\' < 2'E\w{t,X)\\ for ah s > I. 

It is evident that the second integral on the right hand side of (6.8) does not exceed Ms{w 
Moreover, since {Ew^{t,X)) 2 < E\w{t,X)\% s > 2, the following bound is true T,s(w, /) < M, 
We conclude that EY < oo whenever Ms{w) < oo, and 



w 



(6.9) 



EF = E 



sup A(Xi) 



AeA 



i=l 



< ci{s)S.V^J:s{wJ)+2n^/'Ms{w] 



(ii). Upper bound for b = sup;^g^ ll^lloo- Taking into account that I £ £ C (Lemma 4) and 

applying the Holder inequality we get for any x £ X 



\X{x)\ < 



Therefore, in view of Lemma 5(a) 



\w{t,x) -Ew{t,X)\'T{dt) 



\w{-,x)\\s,r 



b = ||A||oo < 2 sup \\w{-,x)\\s,r < 2M,{w). 



(6.10) 

(iii). Upper bound on the "dual" variance a"^ . Since EX{X) = we have 



= sup / \^{x)f{x)v{dx) = sup / 
agaJ /g£ J 



w{t,x)l{t)T{dt) 



f{x)v{dx) 



< 



sup 



w{t,x)l{t)T{dt) 



f{x)iy{dx)< sup [ [ w{t,x)l{t)T{dt) 



f{x)iy{dx). 



The expression on the right hand side is bounded differently depending on the value of s. 
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If s G [1>2) then applying the Holder inequality to the inner integral in the previous expression 
we obtain 

fix)u{dx) < sup \\w{-,x)\\l, < M^{w). 

We remark also that the bound given by (6.11) remains true for all s > 1. This shows, in particular, 
that a is always bounded whenever Mg^w) < oo. 

If s = 2 then we apply inequality (6.1) of Lemma 3 with p = 2 and ^(dx) = i''{dx) = f{x)iy{dx) 
to the integral operator Ii{x) = J 'w{t,x)l{t)T{dt). This leads to the following bound 

(6.12) a^<Ml^y{w). 

If s > 2 then we apply the inequality (6.2) of Lemma 3 with r = 2, p = q = and x = ^' 
to the integral operator Ii{x) = f wit, x)l{t)T{dt). This yields 

(6.13) < C2{s/{s - 1)) Ml^yiw) = C2{s/{s - l))Ml/^^^^^^^y{w). 

Step 3: application of Lemma 1. 

1. Case s G [1,2). Here we have from (6.6), (6.10) and (6.11) 

Ey < { V^5],(t/;, /)} A [An^/'M,{w)^ =: ps{w, /), h< 2M,(w;), < M^{w). 
Therefore applying Lemma 1 we have for all z > 

(6.14) r{||{„|U. > p.(.,/) + .} < exp{- ^^^,^^^ ^ ^Jl^ ^ }' 

where we have used (6.5) in the denominator of the expression inside of the exponent. 

To get the result of the theorem we note that the following trivial upper bound follows from the 
triangle inequality and the statement (a) of Lemma 5: 

UwW s,r <2nMs{w), Vs>l. 

Thus, the probability in (6.14) is equal to zero if z > 2nMs{w); hence we can replace z by 2nMs{w) 
in the denominator of the expression on the right hand side. This leads to the statement of the 
theorem for s G [1,2). 

2. Case s = 2. We have from (6.7), (6.10) and (6.12) 

EY < ^M2{w), b < 2M2{w), < Ml^y{w). 

Thus, for all z > 

IP||IC«;I|2t > \/nM2iw) + z\ < exp < — ^ ^ i, 

and the statement of Theorem 1 is established for s = 2. 

3. Case s > 2. We have from (6.9), (6.10) and (6.13) 

EY <ci{s)[^^s{wJ) + 2n^''Ms{w)\, 

h<2Ms{w)- a^< C2(s/(s-l))il/4/(,+2),.y(^«)- 
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(6.11) 



a 



< 



|u;(t,x)|V(dt) 



Thus, for any z > we get 

P{||e«,||s,r > ci(s)[V^S,(u;,/) + 2n^/^M,H] +z} 
f 

< exp • 



s[W)Z 



where 03(5) is given in (3.2). This completes the proof of the theorem for the case of s > 2. 

We conclude by establishing the inequalities in (3.6). In order to derive the first inequality we ap- 
ply (6.1) of Lemma 3 with p = s/2 > 1, x = i^to the integral operator (t) := J 'w'^{t,x)f{x)i'{dx). 
This yields 

s/2 -1 1/s 



w^{t,x)f{x)i^{dx)] r(dt) 



<M2{w)yf\u 



as claimed. The second inequality in (3.6) follows straightforwardly from the definition of Mp^r,i 



and Mp. 



7. Proofs of Theorem 3 and Corollary 4. 

7.1. Proof of Theorem 3. First we specify the constants appearing in the theorem statement: 



l.e 



29(^+1) 



2c/e 



-r(g+ 1) + 1 jiVz,d(e/8)(2uefi5)nog2(i?5/r5)[l +4S 



cxpj ' 



T, 



2,e 



[ci(s) + 2] ''7Vz,d(6/8) log2 (i?5/rg) [1 + L%] . 



Recall that in view of (3.7), any ^i; € W is represented as w = for some C, ^ li. For every 
< j < log2(-R^/r^) — 1 (without loss of generality we assume that \og2 {R^/r^) is an integer 
number), put 5j = r2^^^, and define the random events 

log2 (RJr^)^l 

A:= fl Aj, Aj := I sup ||e<^2[^]||,/2,, < [2(1 + e)-i5.j\^\. 



j=0 



(i). The following trivial inequality holds 

< snv{\\i^[Q\\s,r-u,Cl{y)U^{ct>[C],f)\ + u.C|(2/) sup C/^ (</.[(], /). 

Therefore 

Esup{||e^[^]||,,.-n,(7)C|(y)t/5((/<[C])}' 



1 



sup 



(7.1) < E 

+ 2'?-^E sup 



+ 

log2 (fi^/rj)-! 
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where Aj denotes the event complementary to Aj, and l(^) is the indicator of the event A. The 
second term on the right hand side is bounded using Theorem 2; our current goal is to bound the 
first and the third term. 

Note that, if the event A holds then for every € Z 

(7.2) C/5(0[C],/)[l + 4ci(s)(l + e)7] > UMC]) > ^/^(.^[C], /) [l - 4ci(5)(l + 6)7] . 
Indeed, in view of (3.9), (3.13) and (3.14) we get 

= udm, f) - ci{s)v^ I tMc]) - ^sim, f) 

(7.3) > Udmj)-Cl{s)^U^2[^]\U/2,r ■ 

Let ( £ Zbe fixed. Since Z^^,, j = 0, . . . , log2 (i?g/rg) — 1, defined in (3.15), form the partition of Z, 
there exists j^, such that C G , • Taking into account that C G , implies C/^(0[C],/) > = 
Sj^-i, we obtain from (7.3) on the event A that 

(7.4) tjdm) > C/5(0[C],/)-2ci(5)(l + e)75,. > f/5(0[C],/)[l-4ci(s)(l + 6)7]. 

Thus, the right hand side inequality in (7.2) is proved. Similarly, we have from (7.3) and (7.4) that 

c/c(0[c]) < udmj) + \udm)-udmj) 

< U{m)+ci{s)^U^2[^]\\s/2,r < UdmJ)[^ + ^cds)il + eh]. 

Thus (7.2) is proved. 

Using the right hand side inequality in (7.2) and applying Theorem 2 we obtain 



E 



sup <! Umlkr - Ue{l)CI{y)Udmy l(^) 

< E sup f ||e<^[^]|U,, - u,C*dy)Udm, /)} 



(7.5) 



< 



C6 

29(^+1)^1 



1 



2qe 



^r(g + l)iVz,d(e/8) [R^C*dl)Y[l + Li%] exp{-y/2}. 



Now we bound the probability P{^j}. Let Zi, 1 = 1,... , N^^di^/^) be a minimal covering of 
by balls of the radius e/8 in the metric d. By definition of Aj we have 



(7.6) 

Note that 



A^z,d(e/8) 

F{A,} < : 



sup 



2,.> [2(l + 6)75,f |. 
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here the first inequahty follows from the condition H^{Za) = ^ij{o) < (70)^, Va G [r^,i2g] and from 
definition of y^; the second inequality holds by the inclusion Zi n "Lsj ^ and because y < y-y. 
Furthermore, by (3.16) and by the above inclusion 

> 5fk^{Zs^) > n Zi), \^ > 6fA^{Zs^) > 6fA^{Zs^ n Zi), 

which leads to 

[2i6j] ' > (z, n ) + VyA^ {Zs^ r\Zi) + yKj^ {is.^ n Zi ) 
= x^{ZinZs^) + c,{y,ZinZs^), 

where C4y, •) := ^A^{-) + yA^(-) [cf. (2.4)]. 

Hence applying Proposition 1 we obtain from (7.6) that 

A^z,d(e/8) 

^ E K U^^[<:]\\s/2,r>{i + e)[>c^{ZinZs^+c*iy,ZinZs^)]} 

00 

< iV2;,d(e/8)[exp{-y} + J^exp{2£:z,d(e2-^) - 9y2^-3fc-2} 

fc=0 

(7.7) < iVz,d(e/8)[l + L(^,y exp{-y/2}, 

where we have used that y > 1- 

Finally, combining (7.1), (7.5), the bound of Theorem 2, and (7.7) we come to the first assertion 
of the theorem. Here we also used that C|(l) < C^{y) because y > I. 

(ii). To prove the second statement we note first the following non-random bound: since ^^(tz;) < 
Ms{w) for all It; G W and s > 2, 

tJs{w) < Ms{w)[cl{s)^/^ + 2n^/'] < [ci{s) + 2]^/^Ms{w), Vu; e W. 

Next, the left inequality in (7.2) implies that for any subset Wo ^ W 

Ac\ sup U^{w) < [1 + 4ci(s)(l + e)7] sup U^{w, /)) =: Aq. 
Therefore P(A) < ^{A), and 



E 



[U{wWl{Ao)} < [ci{s) + 2]'>[V^Ms{w)YF{A). 



Using (7.7) with y = y-f, and definition of the event A we complete the proof. 



7.2. Proof of Corollary 4- First, as in (7.2), we need to bound tJ^{w) := max{U^{w), ^/nM2{'w)} 
from above and from below in terms of U^{w,f) := max{U^{w, f),^/nM2{w)}. Such bounds are 
easily derived from the following trivial fact: for any positive A, B, C and any 6 G (0, 1) 

A{l + 6) > B > A{1- 6) [AvC]{l + 6)>[BvC]>[AvC]{l-6). 

Next, (7.5) remains valid because, by construction, U^{w,f) < JJ^{w,f) and the assumptions, 
allowing to apply Theorem 2 are imposed now on U^i^w, f) instead of U^{w, /). The computations 
leading to (7.7) remain also unchanged if U^{w, f) is replaced by U^{w, /). Note that now and 
are defined via U(^{w,f). ■ 
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8. Proofs of Theorems 4, 5. 

8.1. Proof of Theorem 4- The proof is based on apphcation of Theorem 2. 
Put 



21' 

T4,.: 



^T{q + l)iVz,d(6/8) [1 + Li%] [4w, (l + 4nV2-i/«) 



2</(^+i)y:? 



^r(g + l)iVz,d(e/8) [1 + w^{l + 2^J2^^Jl + Sn-^^ + (8/3)n"i/2}'. 



We have Mp{w) = \\w\\p for ah w G V and p > 1, and (3.9) yields 

An^l'^WwWs, sG[1,2), 



(8.1) [/^(li;,/) 



Therefore, in view of (3.10) 



\/n||u;||2, s = 2. 



^^f4nVv„ SE[1,2), ^^r4ni^w„ 5G[1,2), 
\ ^/nw2, s = 2, \ A/nw2, s = 2. 

It follows from (3.23), the Holder inequality and the formulas for A^{w) and B^{w) immediately 
after (3.9) that 

r ^7n\\w\\l [1,2), [0, [1,2), 

(8.3 Ajiw) < I B.iw) = { , 

I [2f^n/i.+8V^]||«;||i, « = 2, I fllw^lb, ^ = 2. 

In order to apply Theorem 2 we need to check that Xf/^(a) < a for all a G [r^,ii^]. 

Let s G [1,2); here = {C : a/2 < 4nV^||(/,[(^]||^ = 4ni/2||y^||^ < {'i.lfy). By (8.1), 

Assumption (L) and because Zq C Zs(a/4) we have 

^€('A[Ci]-0[C2],/) , An^i^\\m-^m\s ^ 

^^P A(r A ^ - '^^P ... ^ N < a. 

Ci,C26Z„ d((,i,C2j Ci,C2eZ4a/4) d((,i,(,2j 

If s = 2 then = {(" : a/2 < ^/n\\(j)[C,]\\2 = \/^||it'||2 < a}> and again by Assumption (L) 
^^PCi,C2ez„[^^ll'^[Ci] - </'[C2]||2/d(Ci,C2)] < a. Thus X(7^(a) < a for aU a G [r^,R^], and Theorem 2 
can be applied. To this end, we should compute the quantities A^^ and A^^ [see (2.2), (2.3) and 
(2.8)]. 

For s G [1,2) we have by (8.3), definition of Zq and Assumption (L) that 



sup^g(<A[C]) = sup V37^||</.[C]||. = ^ani/2-i/^, 



P A^A /■ \ — sup . , ^ UU 

Ci,C2eZa d((,i,C2j Ci,C2eZ4a/4) d((;i,C,2j 4 

Similarly, if s = 2 then sup^^g^^ ^d^iC]) < a{2f^n^ + 8n-^/2)i/2^ ^^^^^ 

Ci,C26Za alUi<,2j Cl,C26Z2(a) aiU,<,2j 

S X 1/2 
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These computations, and similar computations for A^, yield 



ifn'/'^-i/', »e[l,2), r 0, ,e[i,2), 



Recall that C|(y) = 1 + 2y/yAyi^ + 2j/Ab^ [see (2.9)]. Therefore if for arbitrary z > we set 

^ r sG[l,2), 

then we get C|(y) = 1 + z if s G [1, 2), and 



if s = 2. Then the statements (i)~(ii) follow by application of the moment bound of Theorem 2. 
Observe that C^(l) = l + ^n-^s-iA for s G [1,2), andQ(l) = l + 2[2f^fi^ + 8n-^/'^Y^^ + jn-^/^ 
for s = 2; is given in (8.2). These expressions along with the moment bound of Theorem 2 lead 
to the formulas for Ti^^ and T2^e given in the beginning of the proof. ■ 



8.2. Proof of Theorem 5. First we specify the constants appearing in the statement of the 

1 1/2 

theorem. Put := where ai and Q2 appear in Assumption (W2); then 

(8.4) i9i := [148a^]-i, 1^2 := 5V2ci{s/2)i^alC,/2. 
Define also 

00 

(8.5) h :=8a2ci(5)[C, VC,/2 VI], Li'\/3) := ^ exp {2 W/™(fc-ie)-^/™ - (9/16) 2^-fc-2|^ 

k=l 

and note that lI'V) < 00 because /3 < m. If we set /,(g) := 2''('+i) [2''" - l\-^r{q + l) + l then the 
constants T^^e and Tg^^ appearing in the statement of the theorem are given by 

l + Ll^)(/3)exp{2Cz(/3)}], 
l + Ll^)(/3)exp{2Cz(/3)}" . 

The proof is based on application of Theorem 3 and Corollary 4. These results will be utilized 
with a distance d,, on 3 which is related to the original distance d, and specified below. In order to 
apply Theorem 3 we need to verify its conditions and to compute the quantities A^^ , A^^ , , 
and y^. These computations are routine and tedious. 

We break the proof into steps. 

0*^. Auxiliary results. We begin with preliminary results that will be used in the subsequent proof. 



(8.6) Ts,, := /,(g)(2n,fc,W2)^iVz,d( [fc-^e/8]!/™ ) log 



/C*W2 



Wr, 



.7) T6,e := [ci(s) + 2]''(a,W2) 



lo, 



K*W2 



Wo 
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Lemma 6. Let (3.21) hold and Assumptions (W2) and (W3) he satisfied; then for all w 
and 1 < p < q < oo one has 

Proof. Recall that under (3.21), Mp{w) = \\w\\p for all p > 1. In view of Assumption (W2) for 
any w &V we have 

aiag^'^lltt'lloo [nies{supp(t(;)}] < \\w\\p < ||t/;||oo [nies{supp(t(;)}] 
This inequality, together with Assumption (W3), yields 

^^^'^ll'W^llg ^ n"^/'^||t(;||oo [mes{supp(i(;)}] """^"^ 



nVP||t/;||^[mes{supp(u;)}]^/^ ^~i^^i/p ^i/.^i/pr^i/pu^^ 
[n mes|supp(t(;j|J 



Our next lemma demonstrates that there exists a real number nip £ (0, 1] such that (3.27) holds. 

Lemma 7. Let Assumptions (W) and (L) hold; then for any p > 2, the inequality (3.27) is valid 
with rup = 2/p and Cp = (2a,)l-2/V/*'"^/^ i.e., 

sup sup ^^il^L^^ < (2a.)i-2/V/^-'/'. 

fee[w2,w2] Ci,C2eZ2(fe) [d(Ci,C2)] 

Proof. We obviously have for any p > 2 

||0[Ci]-<^[C2]||, < (||0[Ci]|L + ||'/'[C2]|L)'"'/"(||0[Ci]-<^[C2]||2)'^'. 

Applying Lemma 6 with q = oo and p = 2, we have that sup^g22(fe) ll'^[C]||oo — ba^^~^^'^ for all 
^ £ [w2,W2]. Then in view of Assumption (L) 

n^''^\\4>\Ci\- (l)\C2\\\ 
Ci,Cl6Z2(f)) [d(Ci,C2)]^/P 

as claimed. j 

Lemma 8. Let Assumptions (W) and (L) hold; then for any G 

(8.8) v^||</.[C]||p < A^y^-'/'a, VpG[l,2), 

(8.9) n'/^m\p < a,/.i/P-i/2a, Vp > 2, 

(8.10) ^A^||</''[C]||p < VpG[l,2). 

Proof. By the Holder inequality \\(t)[C]\\p < I^V^'^^'^UMW, then (8.8) holds by definition of Z^. 



The inequality (8.9) follows Lemma 6. In order to prove (8.10) we write [C]||p < ll'/'[C]||c 
note that by Lemma 6 ||(/>[C]||oo < a*^~^/^-y/n||(/>[C] II2 and use (8.9). j 
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l". Notation. Now we establish some notation. Recall that U^(w,f) = ci{s)[y/nT,s{w, f) + 
In^/'Msiw)] and U^{w,f) is given by (3.28). It follows from the definition of C/^C-,/), (3.9) and 
(3.6) that tj^{w,f) > ^/n\\w\\2 and 

U^-^J) < ci{s)[fU^~^/'V^\\w\\2 + 2n^/'\\w\\s] < ci{s)a,[fU^ + 2]V^\\wh, 

where the last inequality is a consequence of Lemma 6. Therefore we put 

= V^W2, = ci(s)a* [f^^^ + 2] y/nw2, 

where w^ and Wp are defined in (3.24). Recall also that Za = {C '■ a/2 < C/g((^[(^], /) < a}. By 
definition of Lf^{w,f) and by the fact that Mp{w) = \\w\\p for all p > 1 we have that C Z2(a) 
for all a £ [r^,R^]] see (3.25). Define the distance 



^.11) d,(Ci,C2) 



d(Ci,C2) V[d(Ci,C2)r% SG[1,4), 

. d(Ci,c2)v[d(Ci,c2)r= v[d(Cl,c2)]"*»/^ s>4, 

where /c* is given in (8.5). Note that d*(-, •) is indeed a distance because by definition nip < 1 for 
all p>2. 

2^. Verification of the condition (3.17). It follows from definition of U^{-, /), (3.9) and (3.6) that 

(8.12) Mimh < mm J) < c^is)[fU'~'/'M\mh+2n'/-^ 

Therefore, by (8.12), Assumption (L) and (3.27) for any Ci)C2 £ 

uMCi] - f) < ci(s) [iU'-^/'Mim - ^mu + 2n^/^\m - </'[C2jiisj 

< ci{sWU'+2Cs]a{d{Ci,C2) V [d(Ci,C2)]"^^}. 

Thus 

UMCi]-^[Uf) . w ,r oi 
Ci,C2 a*l',i,C2j 

and (3.17) is valid, because /c* > ci(s)[f(^^ + 2Cs]; see (8.5) and (8.11). 

3". Computation of Xfj and verification of (3.18). 

We start with bounds on sup^g^^ tl {(jP'lQ) ■ Recall that 

r 4n2/W,/2(</'2[C]), sG (2,4), 

1 ci(s/2)[f^/'V^M2(02[C]) + 2n2/W,/2(<^2^C])], s>A. 
By (8.9), for any C G Z« 

(8.14) n2/^||<^2[C]|U/2 = {n^''\\m\\sf < a*V'/^)-^^||<^[C]|li < alii^^'^^-^a\ Vs > 2, 

(8.15) V^UWh = [n^'^Wmhf < "*V/'«ll0[C]i < «^/U^'/'a2. 

Substituting these bounds in the expression for U{(p'^[Q) and taking into account that /i > 1 in 
view of (W3) we obtain for all s > 2 



(8.13) U{cl>'[C]) 



i.l6) 



sup ?7(02[C]) < ki ti^i~^a^ h := Calais /2)[fl/^ + 2]. 

39 



Now we establish bounds on U{(j)'^[(i] — (^'^[(2]), C1X2 G ^a- 

(a) . First we consider the case s £ (2,4). By the Holder and triangle inequalities we have 

n'^^cP'iCl] - 0'[C2]|L/2 < n2/-l/2[||0[Ci]||2,/(4-s) + ll'^[C2]||2s/(4-s)]^/^ll'^[Cl] " 

Noting that 2s/(4 — s) > 2 and applying (8.9) we have 

n^/'-'/'\m]\\2s/(4-s) < a,f^'/^-'V^mC]\\2 < a,/.2/-ia, VC G Z,. 
Then using Assumption (L) we get 

(8.17) n2/^||0'[Ci] - ^^[C2]\\s/2 < 2a,^L^/'-'a^diCi,C2), ^(1X2 G Z,. 
This along with (8.13) implies that for s G (2,4) 

(8.18) f/(</''[Ci]-0'[C2]) < 8a./i2/--ia2d(Ci,C2), VCi,C2GZa. 

(b) . Now assume that s > 4. We have for Cii C2 G 



2JII2 



< 



ll</'[Cl]||oo + ||</'[C2]||ooJ^/^||<A[Cl]-<^[C2]||2 

(8.19) < 2a,^-V2„2^(^^^^^)^ 

where we used Assumption (L), and (8.9) with p = 00. Furthermore, we have for all Ci) C2 G 



(8.20) 



n'^iAci] - Ac2]\\s/2 < [ii'/'[Ci]iioo + \m2]\\oo]n'/^\mi] - m\\s/2 

< 2C,/2a*f^~'/''a''{d{Ci,C2)r-/\ 



where we have used (8.9) with p = 00 and the definition of nip [see (3.27)]. These inequalities lead 
to the following bound: for all s > 4 



+ 2a 



/z-i/V{d(Ci,C2)v[d(Ci,C2)]' 



s/2 



Y 



U{^\i] - 4>\2]) < 2a,ci{s/2) + 2Us/2 
Combining this with (8.18) we obtain that for all s > 2 

(8.21) ^7(0'[Cl]-</''[C2]) < fe2//^-V{d(Ci,C2)vd™=/2(Ci,C2)}. 

1 /2 

where ^2 := 8a;*ci(5/2)[foo + 2(75/2]- Now using (8.16) and (8.21) we obtain 

x,(a) = sup V sup m^iC]) < ,^-a^ 

and the last bound holds because > kiV k2 [see (8.5)]. Thus the condition (3.18) is valid with 
(8.22) 



^1 i_ 

7 = /isA4 2 , 



Note that the theorem condition fj, > [64cf (s)]'^^^/^'*^^ ensures 7 < [4ci(l + e)] ^ for any e G (0, 1) 
as required in Theorem 3. 
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4''. Bounding Aa^ and Ab^ ■ By the formula for A^{w) given immediately after (3.9), and by (8i 
and (8.9) we have for C G 



Aim]) < 2ci(.)fi[n||<A[C]||L/(.+2) +4^/^ll'A[C]||2||<^[C]||. +8nV^ 

< 2ci(.)fia2[^^/^ + 12a2n-V^] <24a2ci(.)f^a2[;.^/.^^_i/.^^ 

Here we have used that /U > 1, q* > 1 and we write ci{s) instead of 03(5) in the definition of A'^(-) 
because for functions w{t, x) depending on i — x only the constant 02(5) equals one [see (3.2), and 
remark after Lemma 3 in Section 6]. Thus 



sup A^m]) < 5y^^aJ^a[fi'J' + n-^/^^'^ 



In order to bound 74|(0[Ci] — 4>[C2]) we note that for all Ci) C2 G '^a'- 

• by the Holder inequality and by Assumption (L), y^||(;^)[(^i] — (?I)[C2] ||2s/(s+2) ^ o/iy'^d(Ci, C2); 

• by Assumption (L), \/n||(/)[Ci] - </'[C2]||2 < ad(Ci,C2); 
. by (3.27), ni/«||0[Ci] -0[C2]||s < aa[d(Ci, C2)]''"^ 



Therefore 



sup -4g(^[Ci] - 0K2] ^ 5y^f^(c.Vl)a[;.y- + n-V(^^)] 



and A^^ < 5y^^a,foo(C, Vl)[/iy" + n-i/(2s)], Similarly, since B^m]) = |ci(s)||,/.[C]|U, we have 
by (8.9) that A^^ < |ci(s)(Cs V l)a*ri~-'^/'^. Thus we have shown that 



Aa, < H^^l'' + n~^l^^% Ab, < hn-^/', ks := 5ci(s)a.foo(C, V 1). 

These bounds on A^^ and A^^ lead to the definition of C^{y) in (3.29) [see also (2.9)]. Note that 
'do in (3.29) satisfies -do = k-^. 

5". Computation of X^, andy^. 
rst consider 

37n||0[C]||t, and B{(I)^[C]) = 0. Hence, by (8.9) 



(i). First consider the case s G (2,4). RecaU that in this case A'^{(j)'^[(]) = 37n||(/)2[^]||2^2 



supi((/>2[C]) = sup \/37^||(/)[C]||s < V37alfj,'^/'-\^/'^-^/'a\ 

It follows from (8.17) that for any Ci)C2 G 

^ll'/''[Ci]-<^'[C2]||./2 < 2V^a,^?l'-^n^/^-^/'a^d{Ci,Q2). 

Combining these results we obtain that < 2-v/37a*A^^^''~^?^^^^"^^*, and = which, in turn, 
by (3.19) and(8.22) implies that 

y, = l'Xf > (2^/37a^)-V/-l =: y.. 

This explains the definition of the constant "di in (8.4). 
(ii). Now let s > 4; here recall that 

A\<l?[Q]) = 2c,{s/2)il[n\\<t?[C]\\l^^^^^^ 
Observing that for C G 
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(a) . V^W[C]\\2s/{s+i) < a*fJ'l^''a'^ by (8.10) and /i > 1; 

(b) . n^/q<P^[C]\U/2 < aln~y^a^ by (8.14); 

(c) . V^mC]hmC]\\s/2 < a'tfi-^/'^n-y^a^ by (8.15) and (b), 
we obtain 



(8.23) supA{(j)[C]) < 5^/clis/2)^^a,a 

Similarly, for Ci > C2 G we have: 



V^U^iCl] - <P^[C2]\\2s/is+4) < /^*^'V^II</''[Ci]-</''[C2]||2 < 2a,^iV'a^d{Ci,C2) 
nV^||<A2[Ci]-0'[C2]||s/2 < 2C,/,a,n-y'a^d{Ci,C2)r^/' 



V^II'/''[Ci]-<A'[C2]||2||</''[Ci]-</''[C: 



2JIU/2 



1/2 



< 2 



yCV^a,n"i/^a2{d(Ci,C2) V [d(Ci,C2)]"^=/^}, 



where the first line follows from the Holder inequality and (8.19); the second one follows from (8.20); 
and the third line follows from the two previous inequalities. This yields 



sup A A \ - 5fooV2ci(s/2)a*Cs/2a^ /i* +n 



sl2- Then (8.14) gives 



Combining the last inequality with (8.23) we obtain 

Now in order to bound we recall that B{(fP'[C,]) = |ci(s/2)||(/)' 
sup^g^^ ^((A^Kl) < |ci(s/2)a2n-2Ao^ This along with (8.20) leads to 

A^ < hn~^l\ h := lci{s/2)alC,/2. 

Combining these results with (3.19) and taking into account that, by (8.22), 7 = < 1 for 

s > 4, we have 

and an elementary calculation shows that 

yj > /i-'/'(fc4 V [fiV' + n'^/'] =: y,. 

This inequality yields the constant 'd2 appearing in (8.4). 

6". Application of Theorem 3. In order to apply Theorem 3 with the distance d*(-,-) given in 
(8.11) we need to compute the quantity 

00 

4x^P = E«^P {2^^.d. {e2-') - (9/16) 2'k-']. 

k=l 

Note that the entropy number <?z,d, (•) = In {-^z,d*(')} computed with respect to the distance d*. 
Therefore we first express the entropy ^z,d*(") hi terms of the original distance d and then, using 
Assumption (W4), we derive a bound for L 



(^) 

cxp ■ 
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By the definition of the distance d,,, for all 5 G (0, 1) and Cii C2 £ ^ 

where m := 1 A rUg if s G (2, 4) and m := 1 A nis A mg/2 if s > 4. Therefore iV^^d, (<^) ^ 
-^z,d([fc=7^'^]^^™) • In view of Assumption (W4) this yields 



sup \£zAAS)-[K'sr^/"^} < sup {£zA[K'^]'^"')-[K'sr^^"'} 

e(o,i) 56(0,1) 

< sup I £z,d (x) - \ -- 
x6(o,i) 



c6(0,l) 

Thus we obtain that 



00 

4^,)p < exp{2Cz(/3)} 5]exp |2i+W™(A;;ie)-/3/™ _ (g/ig) 2'^A;-2} = exp {2Cz(/3)}l1^)(/3). 



fc=i 



Now the result of the theorem follows from the bounds of Theorem 3. The constants T5 and Tg 
given in the beginning of the proof are obtained from the expressions for Ti ^ and T2^e and bounds of 
Theorem 3. In particular, we used that in view of Lemma 6 -v/nsup^gy^ ll'f^lls ^ n^a^,^^/*"^/^ W2. 



9. Proof of Theorem 7. The proof is based on verification of conditions and application of 
Theorem 5. First we establish auxiliary results that provide the basis for verification of Assump- 
tions (L) and (W). Then, based on these results, we show that all conditions of Theorem 5 are 
fulfilled. This will yield the required result. 

Let K and K' be any functions satisfying Assumptions (Kl) and (K2), and let h^h! be given 
vectors from "H. Let C = {J'^-, h), C = {^': ^')' ^^'^ recall that (pi{C] is the mapping (J^, h) 1— t- n~^Kh. 

Similarly, if Q, K' , Q' are any functions satisfying Assumptions (Kl) and (K2), and if h, h' , t), t)' 
are vectors from V. then z = [{K,h), {Q,t))], z' = [{K' , h'), {Q' ,1)')], and (j)2[z] is the mapping 
[{K,h),{Q,i))]^n~\Kh*Q^). 

1^. Auxiliary results. We begin with auxiliary results about properties of the mappings 0i[C] and 
02 [2^]- The proofs of these results are given in Appendix. 

Define the function 



(9.1) D{x) := e'^^ [x + ^Lyc\/d(e" - 1) + koo(e''" - 1) 



x>0, 



and put 

(9.2) e, := [koo/ki]Z)'(2), 02 := 2'''+\lkf D' (A) , 

where D' is the first derivative of the function D. 

The next lemma states that Assumption (L) is fulfilled for the mappings [C] and z 1— ?• (/'2[2]- 

Lemma 9. Let Assumption (K) hold, and s > 1. If the sets Z^*), i = 1,2, are equipped with the 
distances dg.^(-, •) then Assumption (L) is valid for the mappings Q 1— s- 0i[C] o.nd z 1— t- (I)2[z\- 
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The next three statements provide a basis for verification of Assumption (W). For any h,h' ^% 
let h\/ h' = {hiV h[,. . . ,hdV h'^) and h A h' = {hi A h[, . . . ,hd A h'^). 

Lemma 10. Let Assumptions (Kl) and (K2) hold; then for any p G [l,oo] 

(9.3) Uimp = n-W-^^^'^m^, VCGZ«, 



-i+i/p_ 



(9.4) Ux\C\-<^x\C:\\\y < 



(9.5) 



Observe that Lemma 10 imphes that Assumption (A2) of Section 3.2 is fulfilled for the mappings 
C H> 01 [C] and z 1-^ (1)2 [z] . 

Lemma 11. Let w G Md{l,P) with some P > 0, and let x Mf^ be a point such that w{x) = 
\\w\\cxj > 0; then 



(^) 



xGR'^: \wix)\ > ^\\w\\oo \ 5 



1=1 



Lemma 12. Under Assumption (K) for any p > 1 



Nil <2'^/^'k^n-l(y,vh)-^+l/^ 



(u) ||<^2N|I > 2^ k?n-i (y,vh)^'+'/" , 



{Hi) mes|supp((/)2H)| > (V/ivi,) 



Il-M^lloo ^ ^ \\W\\oo 



2'i+^VdL!ck^ 



{iv) meslt: Mz]{t) > i^\\Mz]\L > 



2'^+2VdL/cko 



mes|supp(02 N)}- 



2''. Verification of conditions of Theorem 5. We check Assumption (W) for the classes of weights 
W(^) and W(2) given by the parametrization 4>i[C] and </>2[2^]. 

First we note that (Wl) is fulfilled both for (pi[C] and (j)2[z] in view of Assumption (Kl). Fur- 
thermore, Assumptions (Kl) and (K2) together with Lemma 11 imply (W2) for i;^>i[C] with 



(9.6) 



_LjcVd_ 

while the statement (iv) of Lemma 12 yield (W2) for (j)2[z] with the constants 



(9.7) 



k? 



2'i+^VdLicka 

Clearly, mes{supp((/>i[C])} ^ V'^imin; hence the condition 



64c? (s) 
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implies (W3) for (j)i[C] with /_f = nV/^min. It follows from the statement (iii) of Lemma 12 that 
Assumption (W3) holds for (f)2[z] with fi = nV/jmin if 

Finally, a standard calculation shows that if £-h{-) is the entropy number of the set T-L measured 
in the distance A-^ [see (3.31)] then for any 6 G (0, 1] 

d 

(9.8) £n{-) < dln{3/6) + ^ (inln [/if ^"//if °] ) . 

i=i 

This result together with (K3) guarantees that Assumption (W4) is fulfilled for the both parametriza- 
tions. 

Now we compute the quantities irip and Cp appearing in (3.27). Although Lemma 7 shows that 
we always can set nip = 2/p, it turns out that under Assumption (K) we can put nip = 1 for 
all p > 2 both for 0i[C] and for (l)2[z]- This leads to weaker conditions on the entropy Sici-) [see 
formulation of Theorem 5]. 

First consider the mapping 0i[C]; here following (3.25) we set 

^^\b) := {C = {K,h) : n'^-'WMQh <b] = {c= {K,h) : {nVhr^l^\\Kh < b}. 
for b G [w^^^w^,^^] where by (3.24) 

(9.9) W^^^ > kl(nT4max)-l/2, W^^) < koo(nl4min)-^/2_ 

By (9.4) of Lemma 10 we have for any p > 2 and ("i = {K, h), C2 = {K' , h!) 

n^'''\\MCi]-HC2]\\p < (nW)"'+'/^I)(dS'^(Ci,C2)), 
and if Cij C2 G ^2^^(^) are such that d[^''(Ci, C2) ^ 2 then by definition of Z2^^(6) 

n^/^ll0i[Ci]-'/'i[C2]||p < [Vki]'^'/^I)'(2)di^)(Ci,C2), 

where we have used that ||i^||2 > ^ ki for all K £ IC, D{0) = 0, and D is monotone increasing. 

If Ci) C2 G '^2^\b) and d^^^(Ci, C2) > 2 then by the triangle inequality, and (9.3) of Lemma 10 

n'/nMCi]-MC2]\\p < n'/nMCi]\\p+n'/nMC2]\\p 

< 2koo(nI4)-i+i/P < koo[6/ki]'~'/^dS^)(Ci,C2). 

These inequalities show that if Z^^^ is equipped with the distance dg^''(-, •) [see (9.2) for definition 
of Oi] then (3.27) holds with 

(9.10) mp = l, Cp = 0ri[koo/ki]'-'/Pl)'(2) < 1, 

because nT^min > 1 (which implies b < koo). 

Now consider the mapping (^2[-z]; following (3.25) we have here 

Z^^\b):={z = [{K,h),{Q,i))]:n'/^\\Mz]\\2<b}, be [wf , w^'^] 
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64cf(s) 



where by the statements (i) and (ii) of Lemma 12 

(9.11) 2"'^/2 k? (nF,^ax)-i/2 < , wf < 2'^/2 (nF;,^,n)-i/2 . 

Note that if z = [{K , h) , {Q , [))] G Zg^^fe) then by the statement (ii) of Lemma 12 we have 
{nVhvt)) ^ < 2^k||"^6^. By this fact and by (9.5) of Lemma 10 we have for zi = [{K, h), {Q, [})], Z2 = 
[{K', h'), iQ', f)')] G ^ such that df ^(zi, zg) < 2 

n'/^||02[^i] -</'2[^2]||p < 2koo[(nW) V (nyf,v^')]"'^'^''^(2df^(^i'^2)) 

< 2'^+2-'^/Pko, [b/kl]'-'^^D'{4)d^^\zi,Z2) 

(2) 

If di ^(zi,Z2) > 2 then using the triangle inequahty and Lemma 12 (i) we have 

n^/^Mzi] - Mz2]\\, < 2'^+^kL[6/k?]^-^/^ < 2'],l[b/kl]'-'/'df\z,,Z2). 

Combining these inequalities we observe that if Z^^^ is equipped with the distance dg^^(-,-) [see 
(9.2)] then (3.27) holds with 

(9.12) m, = l, C, = e^'2''^'~'fkl[kl/k^]'^'^'D'{A)<l. 

We have used that b < 2'^/^k^ because nV^min > 1. Thus (9.10) and (9.12) show that m = 1 and 
the condition /3 < m of Theorem 5 holds if in Assumption (K3) /3a: < 1. 

3*^. Application of Theorem 5. First note that t^Q^ i = 1,2 defined in (3.38) satisfy 
'&o,i ■= 5ci(s)fooa*,j , a*,i := 2/^/a^, i = 1,2, 

where a2,i, i = 1,2 are given in (9.6) and (9.7). This is in accordance with the definition of the 
constant ??o in (3.29) for the parametrizations <^i[C] and (t)2[z\- Then the definition of C^iiu) in 
(3.40) corresponds to (3.29). Following (8.4) we put 

:= /148, ^2,i := 5V2ci(s/2)fooa^,i, i = 1, 2. 

Then the formula for y^*^ appearing in the statement of the theorem is a version of (3.30). 
Now we need to specify the constants T^^^ and Tg^^; see (8.6), (8.7). 
Following (8.5) we set for i = 1,2 

oo 

:= 8cr{s)al„ Lg(/3) := J] exp {2 W(fe,-|e)-^ - (9/16) 2^'fc-2|. 

k=l 



In view of (9.8) and Assumption (K3) we obtain for any f3 G {/3ic, 1) that 

d 

C^mm = sup {£:^(,)^a)(5)-r/'| < C7^ + C/3,rf + 5^(lnln[/irV^"'°]) ' 

d 

Cn^m= sup {8^,,,^^,,{5)-5~^] < 2C^ + 2Cp,d + 2Y,(\n\n[hr^/hf^]) , 
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where we have taken into account that Oi > 1, 02 > I and denoted 



Therefore for i = 1,2 



Cp^d ■■= sup dln(3/5) + 6~^^ -S-^ , /3 G {^ic, 1). 
5e(o,i] 



Ll'l {(3) exp{2C^(.) (/?)} < [1 + An] ' exp{2iC^} inf Lfl (/3) exp{2iC^,,} 

' /3e(/3K;,l) L ' 

and, by Assumption (K3), (9.8) and (3.39) 



exp < I 



exp{zC/c}. 



Finally, substituting these bounds in (8.6) and using (9.9), (9.11), and (3.39) we have that 

T« < {l + An?\l + Bn)f'f\ 

where 



di 



X los 



1 + inf 

/3G(/3k:,1) 



exp <^ I 



L:,^i(/3)exp{2iC^,4 



exp{3iC/c} 
i = 1,2. 



This leads to the first statement of the theorem. The second statement of the theorem follows 
substitution of the above bounds in (8.7) which gives Tg*] < (1 + j4-^)^*(1 + i?-^)T'2*\ where 

di ( /at. i^f^' 



« ._ 



2.e 



[ci(s)+2]^(2^/2a,,k2 



exp < I 



X los 



1 + inf 

/36(/3k:,1) 



Lg(/3)exp{2iC^,,} 



exp{3iC/c} 
, i = l,2. 



10. Proofs of Theorems 8 and 9. 

10.1. Proof of Theorem 8. Let X' = (X, e), and let X[, i = 1, . . . , n be independent copies of 
X' . For any / > 0, x' = (x, n) E x M and t £ T define the function 

With this notation we note that on the event { maxj=i^...^„ |ej| < /} 

n n 

riUt) = Y,w{t,X,)e, = ^w^^\t,Xi) = e^(o(t). 
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and the last equality holds because Eid^'^ (t, X') = 0, for all t G T and / > because the distribution 
of £ is symmetric. Therefore for any 2; > 

^{\\Vu,\\s,r > z] < P{||C^(o|U. >z}+nP{|e| >/}. 

If Assumption (El) is fulfilled then for any z > 

(10.1) ^{\\Vw\\s,r>z] < P{||^^(o|U,r > 4 + ™exp{-6r}. 

If Assumption (E2) is fulfilled then for any z > 

(10.2) P{||77^|U,, >z} < P{||e^(o||.,. > 4 + nPrP. 

In order to bound the first term on the right hand side of (10.1) and (10.2) we repeat the steps in 
the proof of Theorem 1 with w replaced by w^''^ and optimize with respect to the truncation level /. 
For any 2; > we define 

z^ 

Ts{w,f,z) = - , 

l'[u'^{w, f) + ^c^{s)Ms{w)z ' 

where c=k(s) is given in (3.2). 

First consider the case s > 2. Using the same reasoning as in the proof of Theorem 1 we have 
the following upper bound: for all z > 

(10.3) P{ U^o) \\s,r > Qsiw, f) + z}< exp {-[IV l]-^Ts{w, f,z)}. 
Under Assumption (El) if we set 

[b~^Tsiw,f,z)]^ b-^TsiwJ,z)<l, 

[ [b~^Ts{w,f,z)]^ b-^Ts{wJ,z)>l 

then it follows from (10.1) and (10.3) that 

^{\Ms,r > Qs(.wJ)+z} < G«(T,(7i;,/,z)). 

Thus, the first statement of the theorem is proved if s > 2. 
If Assumption (E2) is fulfilled then we choose 

^ _ T^(w,/,z) 

pin {l+p-^Ts{w,f, z)) 

and note that / > 1 for any value of Ts{w, f, z). Then (10.2) and (10.3) imply that 



\llw\\s,T > Qsiw,f) + z} < 



[l+p-^T,{wJ,z)) 



+ nP 



pin {l+p ^Tsiw,f,z)) 



Ts{w,f, z) 



Using the trivial inequality {1 + u) ^ < u -'^ln(l + n), n>Owe get 



\f]w\\s,T > Qs{wJ) +z} <[l +nP] 



pin (l+p ^T^(w,/,z)) 
Ts{w,f,z) 



and, therefore, the second statement of the theorem is proved for the case s > 2. 
If s < 2 then we have similarly to (10.3) that for all z > 

P{|lOolls,r > Qs{wJ) + z}<ei,v{-[l\Jir^Ts{wJ,z)]. 

The same computations as in the case s > 2 lead to the statement of the theorem when s < 2. 
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10.2. Proof of Theorem 9. Put 

k=i •'^ 
Tn,, := (1 + nv) [22^(1 + 6)aw2]^[2''^ - l]"^ exp {CM + {8/ef} (l + exp {2Cz(/3)}4^) ) /^l. 

We note that L^^\ < oo since /? < a/(2 + a) if s < 2, and /? < a/(l + a) if s > 2. Note also that 
the quantity Jg'\-) in the second inequahty of Corollary 1 admits the following bound if y = Gi: 

4\z) < (1 + 7^v) [g^,,{z)] (l + ) J% z > 0. 

If for any G Z we let 
then we have for / G 

Qsimj) < Ur,{m), l^si^icif) < Aiim), ^c,{s)Ms{<p[c]) < Br,{m)- 

Thus, in view of Theorem 8, Assumption 1 holds with U = Urj, A = Arj, B = Brj and g = G^^\ 
Then standard computations show that A^^ = b„ and A^^ = c„. The assertion of the theorem 
follows now from Corollary 1. | 



11. Appendix. 

Proof of Lemma 4- Let 



X • • • X 



X, = X ••• X A'. 



n— times n— times 

Obviously, X^"^^ is a countable dense subset of For any G and t G T put 



i=l 



and let 



^sign[^(i,x("))] 



Note that £ is countable and £ C since, obviously 

s — l 

= 1, Vx(") G 



Note that ^^(^ = = (Xi . . . and therefore, in order to prove the assertion of 

the lemma it is sufficient to show that 



(11.1) 



= sup 
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First let us note that Assumption (Al) implies that for every e > and every x^"^^ S there 
exists x*^") G X^"'^ such that 



:n.2) 



< e. 



Taking into account that £ C M_s_ and using the Holder inequality we obtain from (11.2) that 



sup / /(t)C(t,x("))r(dt) -sup / /(t)e(t,x("))T(dt) 
zefi J i&z J 



(11.3) 
Obviously 

It implies in view of the duality argument that 



< e. 



= / /,(.)(0^(i,x("))r(dt). 



(11.4) 



e(-,x(")) =sup / /(t)e(t,xW)T(dt). 



Using the triangle inequality we obtain from (11.2), (11.3) and (11.4) that for every e > and 
every x^*^) G 

e(-,2;('^0 -sup / /(t)e(t,x("))r(dt) <2e, 
which completes the proof of (11-1) because e > can be chosen arbitrary small. | 

Proof of Lemma 5. First note that for any p > 1 and x & X 



1 



\w{t,x)\PT{dt)+ E\w{t,X)\PT{dt) 



< 2 sup \\w{-,x)\\p^r- 



Here we have used the triangle inequality. Next, for any p > 1 and t £ T 

i/p 



\w{t,x)\Pf{x)u{dx) 



:= [E\iD{t,X)\P]^/P 
< 2[E\w{t,X)\P]^^P =:2 



\w{t, x)|'^/(x)z^(dx) 



Here we used that E|?7 — Er/|P < 2^'E|77|P. Combining both inequalities we have 

Mp{w) < 2Mp{w), 
and the second statement of the lemma is proved. 



50 



Proof of Lemma 9. 

1°. First we establish statement of the lemma for the mapping C, i— s- <^i[C]- For any s > 1 let 
s := s A 2. Following (3.24) and (3.25) and in view of (9.3) we have 

zW(6):={C=(/^,/i)GZ«: [nVhf''-'\\K\\,<b], 6 G [wi^\ wl^^] . 

We note that if C = {K, h) G l}'^\b) then 

(11.5) {nVhf'-^ < kr'fo. 

Let Ci,C2 G 4^^(b) be such that d^^^(Ci,C2) < 2. Applying (9.4) with p = s and using (11.5) we 
get 

(11.6) nV«||</.i[Ci] -0i[C2]||a<kr^6D'(2)d«(Ci,C2) =6d(;)(Ci,C2). 

Here we have taken into account that D'{2) = sup^g[02] |-D'(2;)|, where the function D{-) is given 
in (9.1). If C11C2 G Zs'^\b) are such that d^^^(Ci,C2) > 2, then by the triangle inequality 

(11.7) ni/^||0i[Ci] <26<6dS')(Ci,C2) <m(;)(Ci,C2). 

Thus (11.6) and (11.7) imply that that Assumption (L) holds if Z*^^^ is equipped with the distance 
d^^-*, where we recall that 6*1 = kook]"^D'(2) > 1 [see (9.2)]. 

2". Now we prove the statement of the lemma for the mapping z 1— )• (j)2[z]- By the statements (i) 
and (ii) of Lemma 12 applied with p = s we have 

2^k2(nK...)i/-i < wf < 2'/^kl{nV,^:.f'-\ 

Remind that 

zf(6):={z=[(K,/i),(Q,f))] GZ(2): n'/'\\Mz]l < b} , b G [wi'\wP]. 
li z = \{K, h), (Q, f))] G Zp^(6) then by the statement (ii) of Lemma 12 

(11.8) (nKvf,)'/'"' < 2^kr'6 < 2''/%^. 

Let zi,Z2 G zi^\b) be such that dSi\zi,Z2) < 2. Applying (9.5) with p = 5 and using (11.8) we 
obtain 

(11.9) n'^'\\Mzi]-Uz2]l < b2^+''/\^k^^D'{4)df\zi,Z2). 

(2) (2) 

If zi,Z2 G n '{b) are such that d^^ {zi, Z2) > 2 then we have by the triangle inequality 

(11.10) n^^'\\(l)2[zi] -<^2N||, < 2b<bS^\zi,Z2). 

Thus (11.9) and (11.10) imply that Assumption (L) is valid provided that Z^^-* is equipped with 
the distance df^{; •), where 62 = 22'^+2k^k^2^'(4) > 2'^kook];^D'{A) > 1 [see (9.2)]. | 
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Proof of Lemma 10. 1^. The inequality (9.3) is immediate. We start with the proof of (9.4). 

Since the required bound is symmetric in h and h' , without loss of generality we will assume that 
Vh ^ V/i/. By the triangle inequality in view of Assumption (Kl) we get 

-i+i/p. 



\Kh-K'\\p < m-K\\p + \\K'^-K'^,\\p<V^ 



\K-K'\\p+\\K',, 



-1+1/p 



h 



+ vr}\\K'{./h)-K'{./h')\ 



Ah' 



\K-K'\\^+k, 



iVh' 



Vh. 



Ah' 



(11.11) 



+ {V, 



hVh' 



-1+1/p 



Vh\/h' 
VhAh' 



K'{-[hV h']/h) - K'{-[hV h']/h') 



where h A h' = {hi A h'^, . . . , /\ h'^). The second term of the last inequality is obtained using the 
evident change-of-variables 1 1— )• t/[hy h'] (the division is understood in the coordinate-wise sense) . 

Note that all coordinates of the vectors [h V h']/h and [h V h']/h' are greater or equal to 1. 
Therefore, in view of Assumption (Kl) the integration (or supremum if p = oo) over the whole M'^ 
in \\K' { ■ [h V — K' {■ [h\/ h']/ /i') ||p can be replaced by the integration (supremum) over the 

support of K' . Together with Assumption (Kl) this yields 



K'{-[hy h']/h) -K'{-[hVh']/h') 



1 



hj V h'j 
hj A h'- 



(11.12) < 2'^LicVd{exp{An{h,h')} - l). 
Noting that Vh\/h' /VhAh' < exp {d A-^(/i, /i')| we obtain from (11.11) and (11.12) that 

\\Kh - K'^'Wp 

(11.13) < (Vh^h,) e''^^(^'^'^ [\\K -K'\\^+k^ (^e^^niKh') _ + (^^An{h,h') _ 

Then (9.4) follows from the last inequality and the monotonicity of the function D{-). 

2°. Now we turn to the proof of (9.5). Recall that z = [{K, h), (Q, t})] and z' = [{K', h'), {Q\ \)')]. 
For brevity we also write C,k = {K,h) and C,q = {Q,\]) with evident changes in notation for 
and Cq- 

By the triangle inequality we have 

(11.14) \\Kh * Q„ - * q;,iIp < \\Kh * Qf, - Kh * Q't,'\\p + \\Kh * g;. - * q;,iIp. 

Using the Young inequality (the first statement of Lemma 3), Assumption (Kl) and (11.13) we 
obtain 

\\Kh*Qi,-Kh*Q'Jp < \\KMQi, - Q'^,\\p < koo(V(,vh')"'^'^''^(4'^(CQ,CQ)). 
On the other hand, applying the Young inequality and (11.13) with p = 1, we have 



\Kh*Qi,-Kh*Q'i,,\\p < ||K/,||p||q„-q;,||i < koov; '+'/^z)(dl^^(CQ,CQ)) 



< ko 

< k. 



VhyH'r'^'^''eMd^n{h,h')}D{d['\CQ,CQ)) 



D {2df \z,z')), 



52 



where we have used the definition of A-^(-,-) and monotonicity of the function D(-). Combining 
the last two inequahties we have 



\\Kh*Qi,-Kh*Q'i,,\\p < koo[(W) V(yftvi,')j ' D{2df\z,z')). 

Repeating the previous computations we obtain the same bound for the second term on the right 
hand side of (11.14), namely, 



\\Kh*Q'i,, - K,*Q'i,,\\p < koo [( W) V (yf,vf,') 
Thus, we finally get 



D {2df \z,z')). 



-i+i/p_ 



Kh*Qi,-K,*Q't,,\\p < 2koo (W)v(yt,v(,') "^"'D{2df\z,z')), 



as claimed. 



Proof of Lemma 11. If tt; G 11(^(1, P) then for any 

d 



X G 



4 = 1 



\W\ 



Xi + 



\w\ 



2PVd' 2PVd 



we have by the triangle inequality 



> \'w{x)\ — {ui^x) — w{x)\ > WwWoo — P\x — x\ > -||w||oo- 
This completes the proof. ■ 

Proof of Lemma 12. Recall that 

HAit) = (Kh * Qi,m = j Kh{t - y)Q„(y)dy, t G M^. 

l''. Let J denote the set of indexes j G {1, . . . , d} such that hj < fjj: 

J:= {j€{l,...,d):hj<[)j}. 
Given two arbitrary vectors u,v ^ M.'^, let A[ii, u] and 5[u,f] denote the vectors in with the 



coordinates 



Aj[u,v] 



u 



'01 



With this notation we can write 

1 



VhV^ 



K A 



t — V V 



h ' h 



Q[8 



Uj, j ^ J, 



t — V V 



dv, t G 
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Then changing the variables f i— )• u = (t — v)/{h A h') and setting for brevity t] = {h f\ fl)/(/i V \]) 
(as usual, all operations are understood in the coordinate-wise sense) we come to the formula 



(11.15) 

where we have denoted 
(11.16) 



VhV^ 

1 



t 



u,t/{h V f)) — rytt 



Q(6 



t/{h V f)) — ryn, u 



du 



F{t) := j K(^A[u,t-r]u]^Q(^5[t-r]u,u]^du, t£ 



Now we note some properties of the function F that will be useful in the sequel. First, Assump- 
tion (Kl) implies that the integration over M.'^ in (11.16) can be replaced by the integration over 
[—1/2, 1/2]'^. Indeed, if at least one of the coordinates of u lies outside the interval [—1/2, 1/2] then, 
in view of (Kl), one of the functions K ox Q vanishes. This fact along with Assumption (K2) and 

(11.16) imply that ||-F||oo < k^; in addition, 

(11.17) supp(F) C [-1,1]^^. 
Taking into account these facts and using (11.15) we obtain 

||A',*Q(,||p < {Vhy^r^+^/^Fl, < 2'^/^'k^(y,v(,)"'+'/^ 

and the statement (i) of the lemma is proved. 

To get the assertion (ii) of the lemma we note that 



F{t)dt 



j j K(^A[u,t - T]u]^Q(^6[t - r]u,u]jdu dt 
/K(.)dx /Q(.)dx 



> k?. 



The second equality follows from the fact functions K and Q are integrated over t and over u over 
disjoint sets of components; and the last inequality is a consequence of (K2). Therefore, invoking 
(11.17) we have 



IGII 



(V, 



hVt)) 



\F\\„> [2''Vh 



-i+i/p 



iFlli > 2" 



d(l-p) 



k? {V, 



h\/h) 



-1+1/p 



as claimed in the statement (ii) of the lemma. 

2". Now we turn to the proof of the statements (iii) and (iv) of the lemma. The idea in the proof 
of these statements is to show that F satisfies the Lipschitz condition and then to apply Lemma 11. 

By (11.16) for any x,y we have 



\F{x)-F{y)\ < ko 



sup 



(11.18) 



+ ko 



< i/ckc 



sup 



ne[- 



K (^A[u, X — T]u]^ — K ^ A [u,y — -qu] 
q{5[x - r]u,u]^ - Q(^6[y - r]u,u\ 



IE' 



yj) 
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IE' 



oo\x y\ 



The obtained inequality means that F G Md{l,P) with P = 2Lyckoo; moreover, (11.17) imphes that 
(11.19) 

Applying Lemma 11 and using (11.18) we obtain 



xGM^: F{x)>^\\F\\^} D 



i=l 



\F\ 



F 



Xi + 



2P\/d' 2P^^d 



where, recall, F{x) = ||F|| . Using (11.19) we obviously deduce from (11.15) that 



x: {Kh*Q^){x)> -\\Kh*Q^\\^\ D 



i=l 



Xi{h V - J, , Xi{h V + 



2'i+^PVd' 



2<^+^PVd 



which implies that 

(11.20) mesja;: {Kh * Qi,){x) > ^\\Kh * Qi,\\oo^ > T4vii 
Then the statement (iii) of the lemma follows because 



n d 



mes 



^suj)p{Kh * Qi,)} > mesjx : {Kh * Qh){^) > ^ll-^/i * Qfjlloo}- 



It remains to note that (11.17) implies that mes {supp{Kh * Qf,)} < 2'^ Vhvt,- Therefore by (11.20) 

-I d 



mes : {Kh * Q^){x) > ^\\Kh * Qf,||oo } > 
This completes the proof of the lemma. 



mes{supp(Er/i *Qt))} 
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